提交 3443fd85 编写于 作者: R Roman Donchenko 提交者: OpenCV Buildbot

Merge pull request #984 from albenoit:master

...@@ -17,7 +17,7 @@ if(BUILD_DOCS AND HAVE_SPHINX) ...@@ -17,7 +17,7 @@ if(BUILD_DOCS AND HAVE_SPHINX)
set(OPTIONAL_DOC_LIST "") set(OPTIONAL_DOC_LIST "")
set(OPENCV2_BASE_MODULES core imgproc highgui video calib3d features2d objdetect ml flann gpu photo stitching nonfree contrib legacy) set(OPENCV2_BASE_MODULES core imgproc highgui video calib3d features2d objdetect ml flann gpu photo stitching nonfree contrib legacy bioinspired)
# build lists of modules to be documented # build lists of modules to be documented
set(OPENCV2_MODULES "") set(OPENCV2_MODULES "")
......
此差异已折叠。
...@@ -107,14 +107,16 @@ This retina filter code includes the research contributions of phd/research coll ...@@ -107,14 +107,16 @@ This retina filter code includes the research contributions of phd/research coll
Code tutorial Code tutorial
============= =============
Please refer to the original tutorial source code in file *opencv_folder/samples/cpp/tutorial_code/contrib/retina_tutorial.cpp*. Please refer to the original tutorial source code in file *opencv_folder/samples/cpp/tutorial_code/bioinspired/retina_tutorial.cpp*.
To compile it, assuming OpenCV is correctly installed, use the following command. It requires the opencv_core *(cv::Mat and friends objects management)*, opencv_highgui *(display and image/video read)* and opencv_contrib *(Retina description)* libraries to compile. **Note :** do not forget that the retina model is included in the following namespace : *cv::bioinspired*.
To compile it, assuming OpenCV is correctly installed, use the following command. It requires the opencv_core *(cv::Mat and friends objects management)*, opencv_highgui *(display and image/video read)* and opencv_bioinspired *(Retina description)* libraries to compile.
.. code-block:: cpp .. code-block:: cpp
// compile // compile
gcc retina_tutorial.cpp -o Retina_tuto -lopencv_core -lopencv_highgui -lopencv_contrib gcc retina_tutorial.cpp -o Retina_tuto -lopencv_core -lopencv_highgui -lopencv_bioinspired
// Run commands : add 'log' as a last parameter to apply a spatial log sampling (simulates retina sampling) // Run commands : add 'log' as a last parameter to apply a spatial log sampling (simulates retina sampling)
// run on webcam // run on webcam
...@@ -128,7 +130,7 @@ To compile it, assuming OpenCV is correctly installed, use the following command ...@@ -128,7 +130,7 @@ To compile it, assuming OpenCV is correctly installed, use the following command
Here is a code explanation : Here is a code explanation :
Retina definition is present in the contrib package and a simple include allows to use it Retina definition is present in the bioinspired package and a simple include allows to use it. You can rather use the specific header : *opencv2/bioinspired.hpp* if you prefer but then include the other required openv modules : *opencv2/core.hpp* and *opencv2/highgui.hpp*
.. code-block:: cpp .. code-block:: cpp
...@@ -229,20 +231,20 @@ Once all input parameters are processed, a first image should have been loaded, ...@@ -229,20 +231,20 @@ Once all input parameters are processed, a first image should have been loaded,
return -1; return -1;
} }
Now, everything is ready to run the retina model. I propose here to allocate a retina instance and to manage the eventual log sampling option. The Retina constructor expects at least a cv::Size object that shows the input data size that will have to be managed. One can activate other options such as color and its related color multiplexing strategy (here Bayer multiplexing is chosen using enum cv::RETINA_COLOR_BAYER). If using log sampling, the image reduction factor (smaller output images) and log sampling strengh can be adjusted. Now, everything is ready to run the retina model. I propose here to allocate a retina instance and to manage the eventual log sampling option. The Retina constructor expects at least a cv::Size object that shows the input data size that will have to be managed. One can activate other options such as color and its related color multiplexing strategy (here Bayer multiplexing is chosen using *enum cv::bioinspired::RETINA_COLOR_BAYER*). If using log sampling, the image reduction factor (smaller output images) and log sampling strengh can be adjusted.
.. code-block:: cpp .. code-block:: cpp
// pointer to a retina object // pointer to a retina object
cv::Ptr<cv::Retina> myRetina; cv::Ptr<Retina> myRetina;
// if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision) // if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision)
if (useLogSampling) if (useLogSampling)
{ {
myRetina = cv::createRetina(inputFrame.size(), true, cv::RETINA_COLOR_BAYER, true, 2.0, 10.0); myRetina = cv::bioinspired::createRetina(inputFrame.size(), true, cv::bioinspired::RETINA_COLOR_BAYER, true, 2.0, 10.0);
} }
else// -> else allocate "classical" retina : else// -> else allocate "classical" retina :
myRetina = cv::createRetina(inputFrame.size()); myRetina = cv::bioinspired::createRetina(inputFrame.size());
Once done, the proposed code writes a default xml file that contains the default parameters of the retina. This is useful to make your own config using this template. Here generated template xml file is called *RetinaDefaultParameters.xml*. Once done, the proposed code writes a default xml file that contains the default parameters of the retina. This is useful to make your own config using this template. Here generated template xml file is called *RetinaDefaultParameters.xml*.
......
.. _Table-Of-Content-Bioinspired:
*bioinspired* module. Algorithms inspired from biological models
----------------------------------------------------------------
Here you will learn how to use additional modules of OpenCV defined in the "bioinspired" module.
.. include:: ../../definitions/tocDefinitions.rst
+
.. tabularcolumns:: m{100pt} m{300pt}
.. cssclass:: toctableopencv
=============== ======================================================
|RetinaDemoImg| **Title:** :ref:`Retina_Model`
*Compatibility:* > OpenCV 2.4
*Author:* |Author_AlexB|
You will learn how to process images and video streams with a model of retina filter for details enhancement, spatio-temporal noise removal, luminance correction and spatio-temporal events detection.
=============== ======================================================
.. |RetinaDemoImg| image:: images/retina_TreeHdr_small.jpg
:height: 90pt
:width: 90pt
.. raw:: latex
\pagebreak
.. toctree::
:hidden:
../retina_model/retina_model
...@@ -156,17 +156,17 @@ As always, we would be happy to hear your comments and receive your contribution ...@@ -156,17 +156,17 @@ As always, we would be happy to hear your comments and receive your contribution
:width: 80pt :width: 80pt
:alt: gpu icon :alt: gpu icon
* :ref:`Table-Of-Content-Contrib` * :ref:`Table-Of-Content-Bioinspired`
.. tabularcolumns:: m{100pt} m{300pt} .. tabularcolumns:: m{100pt} m{300pt}
.. cssclass:: toctableopencv .. cssclass:: toctableopencv
=========== ======================================================= ============= =======================================================
|Contrib| Discover additional contribution to OpenCV. |Bioinspired| Algorithms inspired from biological models.
=========== ======================================================= ============= =======================================================
.. |Contrib| image:: images/retina.jpg .. |Bioinspired| image:: images/retina.jpg
:height: 80pt :height: 80pt
:width: 80pt :width: 80pt
:alt: gpu icon :alt: gpu icon
...@@ -219,6 +219,6 @@ As always, we would be happy to hear your comments and receive your contribution ...@@ -219,6 +219,6 @@ As always, we would be happy to hear your comments and receive your contribution
objdetect/table_of_content_objdetect/table_of_content_objdetect objdetect/table_of_content_objdetect/table_of_content_objdetect
ml/table_of_content_ml/table_of_content_ml ml/table_of_content_ml/table_of_content_ml
gpu/table_of_content_gpu/table_of_content_gpu gpu/table_of_content_gpu/table_of_content_gpu
contrib/table_of_content_contrib/table_of_content_contrib bioinspired/table_of_content_bioinspired/table_of_content_bioinspired
ios/table_of_content_ios/table_of_content_ios ios/table_of_content_ios/table_of_content_ios
general/table_of_content_general/table_of_content_general general/table_of_content_general/table_of_content_general
...@@ -52,6 +52,7 @@ ...@@ -52,6 +52,7 @@
#include "opencv2/calib3d.hpp" #include "opencv2/calib3d.hpp"
#include "opencv2/highgui.hpp" #include "opencv2/highgui.hpp"
#include "opencv2/contrib.hpp" #include "opencv2/contrib.hpp"
#include "opencv2/bioinspired.hpp"
#include "opencv2/ml.hpp" #include "opencv2/ml.hpp"
#endif #endif
set(the_description "Biologically inspired algorithms")
ocv_define_module(bioinspired opencv_core OPTIONAL opencv_highgui)
********************************************************************
bioinspired. Biologically inspired vision models and derivated tools
********************************************************************
The module provides biological visual systems models (human visual system and others). It also provides derivated objects that take advantage of those bio-inspired models.
.. toctree::
:maxdepth: 2
Human retina documentation <retina/index>
...@@ -7,6 +7,8 @@ Retina ...@@ -7,6 +7,8 @@ Retina
====== ======
.. ocv:class:: Retina : public Algorithm .. ocv:class:: Retina : public Algorithm
**Note** : do not forget that the retina model is included in the following namespace : *cv::bioinspired*.
Introduction Introduction
++++++++++++ ++++++++++++
...@@ -17,7 +19,7 @@ Class which provides the main controls to the Gipsa/Listic labs human retina mo ...@@ -17,7 +19,7 @@ Class which provides the main controls to the Gipsa/Listic labs human retina mo
* peripheral vision for sensitive transient signals detection (motion and events) : the magnocellular pathway. * peripheral vision for sensitive transient signals detection (motion and events) : the magnocellular pathway.
From a general point of view, this filter whitens the image spectrum and corrects luminance thanks to local adaptation. An other important property is its hability to filter out spatio-temporal noise while enhancing details. From a general point of view, this filter whitens the image spectrum and corrects luminance thanks to local adaptation. An other important property is its hability to filter out spatio-temporal noise while enhancing details.
This model originates from Jeanny Herault work [Herault2010]_. It has been involved in Alexandre Benoit phd and his current research [Benoit2010]_ (he currently maintains this module within OpenCV). It includes the work of other Jeanny's phd student such as [Chaix2007]_ and the log polar transformations of Barthelemy Durette described in Jeanny's book. This model originates from Jeanny Herault work [Herault2010]_. It has been involved in Alexandre Benoit phd and his current research [Benoit2010]_, [Strat2013]_ (he currently maintains this module within OpenCV). It includes the work of other Jeanny's phd student such as [Chaix2007]_ and the log polar transformations of Barthelemy Durette described in Jeanny's book.
**NOTES :** **NOTES :**
...@@ -55,15 +57,20 @@ As observed in this preliminary demo, the retina can be settled up with various ...@@ -55,15 +57,20 @@ As observed in this preliminary demo, the retina can be settled up with various
Here is an overview of the abstract Retina interface, allocate one instance with the *createRetina* functions.:: Here is an overview of the abstract Retina interface, allocate one instance with the *createRetina* functions.::
namespace cv{namespace bioinspired{
class Retina : public Algorithm class Retina : public Algorithm
{ {
public: public:
// parameters setup instance // parameters setup instance
struct RetinaParameters; // this class is detailled later struct RetinaParameters; // this class is detailled later
// main method for input frame processing // main method for input frame processing (all use method, can also perform High Dynamic Range tone mapping)
void run (InputArray inputImage); void run (InputArray inputImage);
// specific method aiming at correcting luminance only (faster High Dynamic Range tone mapping)
void applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage)
// output buffers retreival methods // output buffers retreival methods
// -> foveal color vision details channel with luminance and noise correction // -> foveal color vision details channel with luminance and noise correction
void getParvo (OutputArray retinaOutput_parvo); void getParvo (OutputArray retinaOutput_parvo);
...@@ -99,7 +106,7 @@ Here is an overview of the abstract Retina interface, allocate one instance with ...@@ -99,7 +106,7 @@ Here is an overview of the abstract Retina interface, allocate one instance with
// Allocators // Allocators
cv::Ptr<Retina> createRetina (Size inputSize); cv::Ptr<Retina> createRetina (Size inputSize);
cv::Ptr<Retina> createRetina (Size inputSize, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0); cv::Ptr<Retina> createRetina (Size inputSize, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0);
}} // cv and bioinspired namespaces end
Description Description
+++++++++++ +++++++++++
...@@ -120,12 +127,20 @@ Use : this model can be used basically for spatio-temporal video effects but als ...@@ -120,12 +127,20 @@ Use : this model can be used basically for spatio-temporal video effects but als
* performing motion analysis also taking benefit of the previously cited properties (check out the magnocellular retina channel output, by using the provided **getMagno** methods) * performing motion analysis also taking benefit of the previously cited properties (check out the magnocellular retina channel output, by using the provided **getMagno** methods)
* general image/video sequence description using either one or both channels. An example of the use of Retina in a Bag of Words approach is given in [Strat2013]_.
Literature Literature
========== ==========
For more information, refer to the following papers : For more information, refer to the following papers :
* Model description :
.. [Benoit2010] Benoit A., Caplier A., Durette B., Herault, J., "Using Human Visual System Modeling For Bio-Inspired Low Level Image Processing", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773. DOI <http://dx.doi.org/10.1016/j.cviu.2010.01.011> .. [Benoit2010] Benoit A., Caplier A., Durette B., Herault, J., "Using Human Visual System Modeling For Bio-Inspired Low Level Image Processing", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773. DOI <http://dx.doi.org/10.1016/j.cviu.2010.01.011>
* Model use in a Bag of Words approach :
.. [Strat2013] Strat S., Benoit A., Lambert P., "Retina enhanced SIFT descriptors for video indexing", CBMI2013, Veszprém, Hungary, 2013.
* Please have a look at the reference work of Jeanny Herault that you can read in his book : * Please have a look at the reference work of Jeanny Herault that you can read in his book :
.. [Herault2010] Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891. .. [Herault2010] Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891.
...@@ -138,6 +153,10 @@ This retina filter code includes the research contributions of phd/research coll ...@@ -138,6 +153,10 @@ This retina filter code includes the research contributions of phd/research coll
* take a look at *imagelogpolprojection.hpp* to discover retina spatial log sampling which originates from Barthelemy Durette phd with Jeanny Herault. A Retina / V1 cortex projection is also proposed and originates from Jeanny's discussions. More informations in the above cited Jeanny Heraults's book. * take a look at *imagelogpolprojection.hpp* to discover retina spatial log sampling which originates from Barthelemy Durette phd with Jeanny Herault. A Retina / V1 cortex projection is also proposed and originates from Jeanny's discussions. More informations in the above cited Jeanny Heraults's book.
* Meylan&al work on HDR tone mapping that is implemented as a specific method within the model :
.. [Meylan2007] L. Meylan , D. Alleysson, S. Susstrunk, "A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images", Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816
Demos and experiments ! Demos and experiments !
======================= =======================
...@@ -161,12 +180,14 @@ Take a look at the provided C++ examples provided with OpenCV : ...@@ -161,12 +180,14 @@ Take a look at the provided C++ examples provided with OpenCV :
Then, take a HDR image using bracketing with your camera and generate an OpenEXR image and then process it using the demo. Then, take a HDR image using bracketing with your camera and generate an OpenEXR image and then process it using the demo.
Typical use, supposing that you have the OpenEXR image *memorial.exr* (present in the samples/cpp/ folder) Typical use, supposing that you have the OpenEXR image such as *memorial.exr* (present in the samples/cpp/ folder)
**OpenCVReleaseFolder/bin/OpenEXRimages_HighDynamicRange_Retina_toneMapping memorial.exr** **OpenCVReleaseFolder/bin/OpenEXRimages_HighDynamicRange_Retina_toneMapping memorial.exr [optionnal: 'fast']**
Note that some sliders are made available to allow you to play with luminance compression. Note that some sliders are made available to allow you to play with luminance compression.
If not using the 'fast' option, then, tone mapping is performed using the full retina model [Benoit2010]_. It includes spectral whitening that allows luminance energy to be reduced. When using the 'fast' option, then, a simpler method is used, it is an adaptation of the algorithm presented in [Meylan2007]_. This method gives also good results and is faster to process but it sometimes requires some more parameters adjustement.
Methods description Methods description
=================== ===================
...@@ -176,8 +197,8 @@ Here are detailled the main methods to control the retina model ...@@ -176,8 +197,8 @@ Here are detailled the main methods to control the retina model
Ptr<Retina>::createRetina Ptr<Retina>::createRetina
+++++++++++++++++++++++++ +++++++++++++++++++++++++
.. ocv:function:: Ptr<Retina> createRetina(Size inputSize) .. ocv:function:: Ptr<cv::bioinspired::Retina> createRetina(Size inputSize)
.. ocv:function:: Ptr<Retina> createRetina(Size inputSize, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod = RETINA_COLOR_BAYER, const bool useRetinaLogSampling = false, const double reductionFactor = 1.0, const double samplingStrenght = 10.0 ) .. ocv:function:: Ptr<cv::bioinspired::Retina> createRetina(Size inputSize, const bool colorMode, cv::bioinspired::RETINA_COLORSAMPLINGMETHOD colorSamplingMethod = cv::bioinspired::RETINA_COLOR_BAYER, const bool useRetinaLogSampling = false, const double reductionFactor = 1.0, const double samplingStrenght = 10.0 )
Constructors from standardized interfaces : retreive a smart pointer to a Retina instance Constructors from standardized interfaces : retreive a smart pointer to a Retina instance
...@@ -185,11 +206,11 @@ Ptr<Retina>::createRetina ...@@ -185,11 +206,11 @@ Ptr<Retina>::createRetina
:param colorMode: the chosen processing mode : with or without color processing :param colorMode: the chosen processing mode : with or without color processing
:param colorSamplingMethod: specifies which kind of color sampling will be used : :param colorSamplingMethod: specifies which kind of color sampling will be used :
* RETINA_COLOR_RANDOM: each pixel position is either R, G or B in a random choice * cv::bioinspired::RETINA_COLOR_RANDOM: each pixel position is either R, G or B in a random choice
* RETINA_COLOR_DIAGONAL: color sampling is RGBRGBRGB..., line 2 BRGBRGBRG..., line 3, GBRGBRGBR... * cv::bioinspired::RETINA_COLOR_DIAGONAL: color sampling is RGBRGBRGB..., line 2 BRGBRGBRG..., line 3, GBRGBRGBR...
* RETINA_COLOR_BAYER: standard bayer sampling * cv::bioinspired::RETINA_COLOR_BAYER: standard bayer sampling
:param useRetinaLogSampling: activate retina log sampling, if true, the 2 following parameters can be used :param useRetinaLogSampling: activate retina log sampling, if true, the 2 following parameters can be used
:param reductionFactor: only usefull if param useRetinaLogSampling=true, specifies the reduction factor of the output frame (as the center (fovea) is high resolution and corners can be underscaled, then a reduction of the output is allowed without precision leak :param reductionFactor: only usefull if param useRetinaLogSampling=true, specifies the reduction factor of the output frame (as the center (fovea) is high resolution and corners can be underscaled, then a reduction of the output is allowed without precision leak
...@@ -275,7 +296,7 @@ Retina::printSetup ...@@ -275,7 +296,7 @@ Retina::printSetup
Outputs a string showing the used parameters setup Outputs a string showing the used parameters setup
:return: a string which contains formatted parameters information :return: a string which contains formated parameters information
Retina::run Retina::run
+++++++++++ +++++++++++
...@@ -286,6 +307,18 @@ Retina::run ...@@ -286,6 +307,18 @@ Retina::run
:param inputImage: the input Mat image to be processed, can be gray level or BGR coded in any format (from 8bit to 16bits) :param inputImage: the input Mat image to be processed, can be gray level or BGR coded in any format (from 8bit to 16bits)
Retina::applyFastToneMapping
++++++++++++++++++++++++++++
.. ocv:function:: void Retina::applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage)
Method which processes an image in the aim to correct its luminance : correct backlight problems, enhance details in shadows. This method is designed to perform High Dynamic Range image tone mapping (compress >8bit/pixel images to 8bit/pixel). This is a simplified version of the Retina Parvocellular model (simplified version of the run/getParvo methods call) since it does not include the spatio-temporal filter modelling the Outer Plexiform Layer of the retina that performs spectral whitening and many other stuff. However, it works great for tone mapping and in a faster way.
Check the demos and experiments section to see examples and the way to perform tone mapping using the original retina model and the method.
:param inputImage: the input image to process (should be coded in float format : CV_32F, CV_32FC1, CV_32F_C3, CV_32F_C4, the 4th channel won't be considered).
:param outputToneMappedImage: the output 8bit/channel tone mapped image (CV_8U or CV_8UC3 format).
Retina::setColorSaturation Retina::setColorSaturation
++++++++++++++++++++++++++ ++++++++++++++++++++++++++
......
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of the copyright holders may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#ifndef __OPENCV_BIOINSPIRED_HPP__
#define __OPENCV_BIOINSPIRED_HPP__
#include "opencv2/core.hpp"
#include "opencv2/bioinspired/retina.hpp"
#include "opencv2/bioinspired/retinafasttonemapping.hpp"
#endif
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
// Copyright (C) 2013, OpenCV Foundation, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of the copyright holders may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#ifdef __OPENCV_BUILD
#error this is a compatibility header which should not be used inside the OpenCV library
#endif
#include "opencv2/bioinspired.hpp"
...@@ -6,12 +6,12 @@ ...@@ -6,12 +6,12 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
** **
** Creation - enhancement process 2007-2011 ** Creation - enhancement process 2007-2013
** Author: Alexandre Benoit (benoit.alexandre.vision@gmail.com), LISTIC lab, Annecy le vieux, France ** Author: Alexandre Benoit (benoit.alexandre.vision@gmail.com), LISTIC lab, Annecy le vieux, France
** **
** Theses algorithm have been developped by Alexandre BENOIT since his thesis with Alice Caplier at Gipsa-Lab (www.gipsa-lab.inpg.fr) and the research he pursues at LISTIC Lab (www.listic.univ-savoie.fr). ** Theses algorithm have been developped by Alexandre BENOIT since his thesis with Alice Caplier at Gipsa-Lab (www.gipsa-lab.inpg.fr) and the research he pursues at LISTIC Lab (www.listic.univ-savoie.fr).
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -62,8 +62,8 @@ ...@@ -62,8 +62,8 @@
** the use of this software, even if advised of the possibility of such damage. ** the use of this software, even if advised of the possibility of such damage.
*******************************************************************************/ *******************************************************************************/
#ifndef __OPENCV_CONTRIB_RETINA_HPP__ #ifndef __OPENCV_BIOINSPIRED_RETINA_HPP__
#define __OPENCV_CONTRIB_RETINA_HPP__ #define __OPENCV_BIOINSPIRED_RETINA_HPP__
/* /*
* Retina.hpp * Retina.hpp
...@@ -73,13 +73,12 @@ ...@@ -73,13 +73,12 @@
*/ */
#include "opencv2/core.hpp" // for all OpenCV core functionalities access, including cv::Exception support #include "opencv2/core.hpp" // for all OpenCV core functionalities access, including cv::Exception support
#include <valarray>
namespace cv
{
enum RETINA_COLORSAMPLINGMETHOD namespace cv{
{ namespace bioinspired{
enum {
RETINA_COLOR_RANDOM, //!< each pixel position is either R, G or B in a random choice RETINA_COLOR_RANDOM, //!< each pixel position is either R, G or B in a random choice
RETINA_COLOR_DIAGONAL,//!< color sampling is RGBRGBRGB..., line 2 BRGBRGBRG..., line 3, GBRGBRGBR... RETINA_COLOR_DIAGONAL,//!< color sampling is RGBRGBRGB..., line 2 BRGBRGBRG..., line 3, GBRGBRGBR...
RETINA_COLOR_BAYER//!< standard bayer sampling RETINA_COLOR_BAYER//!< standard bayer sampling
...@@ -241,6 +240,14 @@ public: ...@@ -241,6 +240,14 @@ public:
*/ */
virtual void run(InputArray inputImage)=0; virtual void run(InputArray inputImage)=0;
/**
* method that applies a luminance correction (initially High Dynamic Range (HDR) tone mapping) using only the 2 local adaptation stages of the retina parvo channel : photoreceptors level and ganlion cells level. Spatio temporal filtering is applied but limited to temporal smoothing and eventually high frequencies attenuation. This is a lighter method than the one available using the regular run method. It is then faster but it does not include complete temporal filtering nor retina spectral whitening. Then, it can have a more limited effect on images with a very high dynamic range. This is an adptation of the original still image HDR tone mapping algorithm of David Alleyson, Sabine Susstruck and Laurence Meylan's work, please cite:
* -> Meylan L., Alleysson D., and Susstrunk S., A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images, Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816
@param inputImage the input image to process RGB or gray levels
@param outputToneMappedImage the output tone mapped image
*/
virtual void applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage)=0;
/** /**
* accessor of the details channel of the retina (models foveal vision) * accessor of the details channel of the retina (models foveal vision)
* @param retinaOutput_parvo : the output buffer (reallocated if necessary), this output is rescaled for standard 8bits image processing use in OpenCV * @param retinaOutput_parvo : the output buffer (reallocated if necessary), this output is rescaled for standard 8bits image processing use in OpenCV
...@@ -295,8 +302,9 @@ public: ...@@ -295,8 +302,9 @@ public:
virtual void activateContoursProcessing(const bool activate)=0; virtual void activateContoursProcessing(const bool activate)=0;
}; };
CV_EXPORTS Ptr<Retina> createRetina(Size inputSize); CV_EXPORTS Ptr<Retina> createRetina(Size inputSize);
CV_EXPORTS Ptr<Retina> createRetina(Size inputSize, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0); CV_EXPORTS Ptr<Retina> createRetina(Size inputSize, const bool colorMode, int colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0);
}
#endif /* __OPENCV_CONTRIB_RETINA_HPP__ */
}
}
#endif /* __OPENCV_BIOINSPIRED_RETINA_HPP__ */
/*#******************************************************************************
** IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
**
** By downloading, copying, installing or using the software you agree to this license.
** If you do not agree to this license, do not download, install,
** copy or use the software.
**
**
** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
**
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
**
** Creation - enhancement process 2007-2013
** Author: Alexandre Benoit (benoit.alexandre.vision@gmail.com), LISTIC lab, Annecy le vieux, France
**
** Theses algorithm have been developped by Alexandre BENOIT since his thesis with Alice Caplier at Gipsa-Lab (www.gipsa-lab.inpg.fr) and the research he pursues at LISTIC Lab (www.listic.univ-savoie.fr).
** Refer to the following research paper for more information:
** Benoit A., Caplier A., Durette B., Herault, J., "USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773, DOI: http://dx.doi.org/10.1016/j.cviu.2010.01.011
** This work have been carried out thanks to Jeanny Herault who's research and great discussions are the basis of all this work, please take a look at his book:
** Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891.
**
**
**
**
**
** This class is based on image processing tools of the author and already used within the Retina class (this is the same code as method retina::applyFastToneMapping, but in an independent class, it is ligth from a memory requirement point of view). It implements an adaptation of the efficient tone mapping algorithm propose by David Alleyson, Sabine Susstruck and Laurence Meylan's work, please cite:
** -> Meylan L., Alleysson D., and Susstrunk S., A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images, Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816
**
**
** License Agreement
** For Open Source Computer Vision Library
**
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
**
** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
**
** Third party copyrights are property of their respective owners.
**
** Redistribution and use in source and binary forms, with or without modification,
** are permitted provided that the following conditions are met:
**
** * Redistributions of source code must retain the above copyright notice,
** this list of conditions and the following disclaimer.
**
** * Redistributions in binary form must reproduce the above copyright notice,
** this list of conditions and the following disclaimer in the documentation
** and/or other materials provided with the distribution.
**
** * The name of the copyright holders may not be used to endorse or promote products
** derived from this software without specific prior written permission.
**
** This software is provided by the copyright holders and contributors "as is" and
** any express or implied warranties, including, but not limited to, the implied
** warranties of merchantability and fitness for a particular purpose are disclaimed.
** In no event shall the Intel Corporation or contributors be liable for any direct,
** indirect, incidental, special, exemplary, or consequential damages
** (including, but not limited to, procurement of substitute goods or services;
** loss of use, data, or profits; or business interruption) however caused
** and on any theory of liability, whether in contract, strict liability,
** or tort (including negligence or otherwise) arising in any way out of
** the use of this software, even if advised of the possibility of such damage.
*******************************************************************************/
#ifndef __OPENCV_BIOINSPIRED_RETINAFASTTONEMAPPING_HPP__
#define __OPENCV_BIOINSPIRED_RETINAFASTTONEMAPPING_HPP__
/*
* retinafasttonemapping.hpp
*
* Created on: May 26, 2013
* Author: Alexandre Benoit
*/
#include "opencv2/core.hpp" // for all OpenCV core functionalities access, including cv::Exception support
namespace cv{
namespace bioinspired{
/**
* @class RetinaFastToneMappingImpl a wrapper class which allows the tone mapping algorithm of Meylan&al(2007) to be used with OpenCV.
* This algorithm is already implemented in thre Retina class (retina::applyFastToneMapping) but used it does not require all the retina model to be allocated. This allows a light memory use for low memory devices (smartphones, etc.
* As a summary, these are the model properties:
* => 2 stages of local luminance adaptation with a different local neighborhood for each.
* => first stage models the retina photorecetors local luminance adaptation
* => second stage models th ganglion cells local information adaptation
* => compared to the initial publication, this class uses spatio-temporal low pass filters instead of spatial only filters.
* ====> this can help noise robustness and temporal stability for video sequence use cases.
* for more information, read to the following papers :
* Meylan L., Alleysson D., and Susstrunk S., A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images, Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816Benoit A., Caplier A., Durette B., Herault, J., "USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773, DOI: http://dx.doi.org/10.1016/j.cviu.2010.01.011
* regarding spatio-temporal filter and the bigger retina model :
* Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891.
*/
class CV_EXPORTS RetinaFastToneMapping : public Algorithm
{
public:
/**
* method that applies a luminance correction (initially High Dynamic Range (HDR) tone mapping) using only the 2 local adaptation stages of the retina parvocellular channel : photoreceptors level and ganlion cells level. Spatio temporal filtering is applied but limited to temporal smoothing and eventually high frequencies attenuation. This is a lighter method than the one available using the regular retina::run method. It is then faster but it does not include complete temporal filtering nor retina spectral whitening. Then, it can have a more limited effect on images with a very high dynamic range. This is an adptation of the original still image HDR tone mapping algorithm of David Alleyson, Sabine Susstruck and Laurence Meylan's work, please cite:
* -> Meylan L., Alleysson D., and Susstrunk S., A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images, Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816
@param inputImage the input image to process RGB or gray levels
@param outputToneMappedImage the output tone mapped image
*/
virtual void applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage)=0;
/**
* setup method that updates tone mapping behaviors by adjusing the local luminance computation area
* @param photoreceptorsNeighborhoodRadius the first stage local adaptation area
* @param ganglioncellsNeighborhoodRadius the second stage local adaptation area
* @param meanLuminanceModulatorK the factor applied to modulate the meanLuminance information (default is 1, see reference paper)
*/
virtual void setup(const float photoreceptorsNeighborhoodRadius=3.f, const float ganglioncellsNeighborhoodRadius=1.f, const float meanLuminanceModulatorK=1.f)=0;
};
CV_EXPORTS Ptr<RetinaFastToneMapping> createRetinaFastToneMapping(Size inputSize);
}
}
#endif /* __OPENCV_BIOINSPIRED_RETINAFASTTONEMAPPING_HPP__ */
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -72,7 +72,8 @@ ...@@ -72,7 +72,8 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
// @author Alexandre BENOIT, benoit.alexandre.vision@gmail.com, LISTIC : www.listic.univ-savoie.fr Gipsa-Lab, France: www.gipsa-lab.inpg.fr/ // @author Alexandre BENOIT, benoit.alexandre.vision@gmail.com, LISTIC : www.listic.univ-savoie.fr Gipsa-Lab, France: www.gipsa-lab.inpg.fr/
////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////
...@@ -883,4 +884,5 @@ void BasicRetinaFilter::_verticalAnticausalFilter_Irregular_multGain(float *outp ...@@ -883,4 +884,5 @@ void BasicRetinaFilter::_verticalAnticausalFilter_Irregular_multGain(float *outp
} }
} }
} }// end of namespace bioinspired
}// end of namespace cv
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -113,6 +113,8 @@ ...@@ -113,6 +113,8 @@
//#define __BASIC_RETINA_ELEMENT_DEBUG //#define __BASIC_RETINA_ELEMENT_DEBUG
namespace cv namespace cv
{
namespace bioinspired
{ {
class BasicRetinaFilter class BasicRetinaFilter
{ {
...@@ -287,7 +289,7 @@ namespace cv ...@@ -287,7 +289,7 @@ namespace cv
* @param maxInputValue: the maximum amplitude value measured after local adaptation processing (c.f. function runFilter_LocalAdapdation & runFilter_LocalAdapdation_autonomous) * @param maxInputValue: the maximum amplitude value measured after local adaptation processing (c.f. function runFilter_LocalAdapdation & runFilter_LocalAdapdation_autonomous)
* @param meanLuminance: the a priori meann luminance of the input data (should be 128 for 8bits images but can vary greatly in case of High Dynamic Range Images (HDRI) * @param meanLuminance: the a priori meann luminance of the input data (should be 128 for 8bits images but can vary greatly in case of High Dynamic Range Images (HDRI)
*/ */
void setV0CompressionParameterToneMapping(const float v0, const float maxInputValue, const float meanLuminance=128.0f){ _v0=v0*maxInputValue; _localLuminanceFactor=1.0f; _localLuminanceAddon=meanLuminance*_v0; _maxInputValue=maxInputValue;}; void setV0CompressionParameterToneMapping(const float v0, const float maxInputValue, const float meanLuminance=128.0f){ _v0=v0*maxInputValue; _localLuminanceFactor=1.0f; _localLuminanceAddon=meanLuminance*v0; _maxInputValue=maxInputValue;};
/** /**
* update compression parameters while keeping v0 parameter value * update compression parameters while keeping v0 parameter value
...@@ -650,7 +652,6 @@ namespace cv ...@@ -650,7 +652,6 @@ namespace cv
}; };
} }// end of namespace bioinspired
}// end of namespace cv
#endif #endif
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -72,7 +72,8 @@ ...@@ -72,7 +72,8 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
// constructor // constructor
ImageLogPolProjection::ImageLogPolProjection(const unsigned int nbRows, const unsigned int nbColumns, const PROJECTIONTYPE projection, const bool colorModeCapable) ImageLogPolProjection::ImageLogPolProjection(const unsigned int nbRows, const unsigned int nbColumns, const PROJECTIONTYPE projection, const bool colorModeCapable)
:BasicRetinaFilter(nbRows, nbColumns), :BasicRetinaFilter(nbRows, nbColumns),
...@@ -446,4 +447,5 @@ std::valarray<float> &ImageLogPolProjection::runProjection(const std::valarray<f ...@@ -446,4 +447,5 @@ std::valarray<float> &ImageLogPolProjection::runProjection(const std::valarray<f
return _sampledFrame; return _sampledFrame;
} }
} }// end of namespace bioinspired
}// end of namespace cv
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -103,6 +103,8 @@ ...@@ -103,6 +103,8 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
class ImageLogPolProjection:public BasicRetinaFilter class ImageLogPolProjection:public BasicRetinaFilter
{ {
...@@ -236,5 +238,6 @@ private: ...@@ -236,5 +238,6 @@ private:
}; };
} }// end of namespace bioinspired
}// end of namespace cv
#endif /*IMAGELOGPOLPROJECTION_H_*/ #endif /*IMAGELOGPOLPROJECTION_H_*/
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -72,6 +72,8 @@ ...@@ -72,6 +72,8 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
// Constructor and Desctructor of the OPL retina filter // Constructor and Desctructor of the OPL retina filter
MagnoRetinaFilter::MagnoRetinaFilter(const unsigned int NBrows, const unsigned int NBcolumns) MagnoRetinaFilter::MagnoRetinaFilter(const unsigned int NBrows, const unsigned int NBcolumns)
:BasicRetinaFilter(NBrows, NBcolumns, 2), :BasicRetinaFilter(NBrows, NBcolumns, 2),
...@@ -206,6 +208,5 @@ const std::valarray<float> &MagnoRetinaFilter::runFilter(const std::valarray<flo ...@@ -206,6 +208,5 @@ const std::valarray<float> &MagnoRetinaFilter::runFilter(const std::valarray<flo
return (*_magnoYOutput); return (*_magnoYOutput);
} }
} }// end of namespace bioinspired
}// end of namespace cv
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -99,7 +99,8 @@ ...@@ -99,7 +99,8 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
class MagnoRetinaFilter: public BasicRetinaFilter class MagnoRetinaFilter: public BasicRetinaFilter
{ {
public: public:
...@@ -238,8 +239,7 @@ namespace cv ...@@ -238,8 +239,7 @@ namespace cv
#endif #endif
}; };
} }// end of namespace bioinspired
}// end of namespace cv
#endif /*MagnoRetinaFilter_H_*/ #endif /*MagnoRetinaFilter_H_*/
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -73,6 +73,8 @@ ...@@ -73,6 +73,8 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////
// OPL RETINA FILTER // OPL RETINA FILTER
////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////
...@@ -227,5 +229,5 @@ void ParvoRetinaFilter::_OPL_OnOffWaysComputing() // WARNING : this method requi ...@@ -227,5 +229,5 @@ void ParvoRetinaFilter::_OPL_OnOffWaysComputing() // WARNING : this method requi
} }
#endif #endif
} }
} }// end of namespace bioinspired
}// end of namespace cv
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -103,6 +103,8 @@ ...@@ -103,6 +103,8 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
//retina classes that derivate from the Basic Retrina class //retina classes that derivate from the Basic Retrina class
class ParvoRetinaFilter: public BasicRetinaFilter class ParvoRetinaFilter: public BasicRetinaFilter
{ {
...@@ -256,6 +258,6 @@ private: ...@@ -256,6 +258,6 @@ private:
#endif #endif
}; };
} }// end of namespace bioinspired
}// end of namespace cv
#endif #endif
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// Intel License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2000, Intel Corporation, all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of Intel Corporation may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#include "precomp.hpp"
/* End of file. */
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of the copyright holders may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#ifndef __OPENCV_PRECOMP_H__
#define __OPENCV_PRECOMP_H__
#include "opencv2/bioinspired.hpp"
#include "opencv2/core/utility.hpp"
#include "opencv2/core/private.hpp"
#include <valarray>
namespace cv
{
// special function to get pointer to constant valarray elements, since
// simple &arr[0] does not compile on VS2005/VS2008.
template<typename T> inline const T* get_data(const std::valarray<T>& arr)
{ return &((std::valarray<T>&)arr)[0]; }
}
#endif
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -72,9 +72,12 @@ ...@@ -72,9 +72,12 @@
#include "retinafilter.hpp" #include "retinafilter.hpp"
#include <cstdio> #include <cstdio>
#include <sstream> #include <sstream>
#include <valarray>
namespace cv namespace cv
{ {
namespace bioinspired
{
class RetinaImpl : public Retina class RetinaImpl : public Retina
{ {
...@@ -94,7 +97,7 @@ public: ...@@ -94,7 +97,7 @@ public:
* @param reductionFactor: only usefull if param useRetinaLogSampling=true, specifies the reduction factor of the output frame (as the center (fovea) is high resolution and corners can be underscaled, then a reduction of the output is allowed without precision leak * @param reductionFactor: only usefull if param useRetinaLogSampling=true, specifies the reduction factor of the output frame (as the center (fovea) is high resolution and corners can be underscaled, then a reduction of the output is allowed without precision leak
* @param samplingStrenght: only usefull if param useRetinaLogSampling=true, specifies the strenght of the log scale that is applied * @param samplingStrenght: only usefull if param useRetinaLogSampling=true, specifies the strenght of the log scale that is applied
*/ */
RetinaImpl(Size inputSize, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0); RetinaImpl(Size inputSize, const bool colorMode, int colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0);
virtual ~RetinaImpl(); virtual ~RetinaImpl();
/** /**
...@@ -196,6 +199,14 @@ public: ...@@ -196,6 +199,14 @@ public:
*/ */
void run(InputArray inputImage); void run(InputArray inputImage);
/**
* method that applies a luminance correction (initially High Dynamic Range (HDR) tone mapping) using only the 2 local adaptation stages of the retina parvo channel : photoreceptors level and ganlion cells level. Spatio temporal filtering is applied but limited to temporal smoothing and eventually high frequencies attenuation. This is a lighter method than the one available using the regular run method. It is then faster but it does not include complete temporal filtering nor retina spectral whitening. This is an adptation of the original still image HDR tone mapping algorithm of David Alleyson, Sabine Susstruck and Laurence Meylan's work, please cite:
* -> Meylan L., Alleysson D., and Susstrunk S., A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images, Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816
@param inputImage the input image to process RGB or gray levels
@param outputToneMappedImage the output tone mapped image
*/
void applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage);
/** /**
* accessor of the details channel of the retina (models foveal vision) * accessor of the details channel of the retina (models foveal vision)
* @param retinaOutput_parvo : the output buffer (reallocated if necessary), this output is rescaled for standard 8bits image processing use in OpenCV * @param retinaOutput_parvo : the output buffer (reallocated if necessary), this output is rescaled for standard 8bits image processing use in OpenCV
...@@ -259,8 +270,11 @@ private: ...@@ -259,8 +270,11 @@ private:
// pointer to retina model // pointer to retina model
RetinaFilter* _retinaFilter; //!< the pointer to the retina module, allocated with instance construction RetinaFilter* _retinaFilter; //!< the pointer to the retina module, allocated with instance construction
//! private method called by constructors, gathers their parameters and use them in a unified way
void _init(const Size inputSize, const bool colorMode, int colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0);
/** /**
* exports a valarray buffer outing from HVStools objects to a cv::Mat in CV_8UC1 (gray level picture) or CV_8UC3 (color) format * exports a valarray buffer outing from bioinspired objects to a cv::Mat in CV_8UC1 (gray level picture) or CV_8UC3 (color) format
* @param grayMatrixToConvert the valarray to export to OpenCV * @param grayMatrixToConvert the valarray to export to OpenCV
* @param nbRows : the number of rows of the valarray flatten matrix * @param nbRows : the number of rows of the valarray flatten matrix
* @param nbColumns : the number of rows of the valarray flatten matrix * @param nbColumns : the number of rows of the valarray flatten matrix
...@@ -270,22 +284,20 @@ private: ...@@ -270,22 +284,20 @@ private:
void _convertValarrayBuffer2cvMat(const std::valarray<float> &grayMatrixToConvert, const unsigned int nbRows, const unsigned int nbColumns, const bool colorMode, OutputArray outBuffer); void _convertValarrayBuffer2cvMat(const std::valarray<float> &grayMatrixToConvert, const unsigned int nbRows, const unsigned int nbColumns, const bool colorMode, OutputArray outBuffer);
/** /**
* * convert a cv::Mat to a valarray buffer in float format
* @param inputMatToConvert : the OpenCV cv::Mat that has to be converted to gray or RGB valarray buffer that will be processed by the retina model * @param inputMatToConvert : the OpenCV cv::Mat that has to be converted to gray or RGB valarray buffer that will be processed by the retina model
* @param outputValarrayMatrix : the output valarray * @param outputValarrayMatrix : the output valarray
* @return the input image color mode (color=true, gray levels=false) * @return the input image color mode (color=true, gray levels=false)
*/ */
bool _convertCvMat2ValarrayBuffer(InputArray inputMatToConvert, std::valarray<float> &outputValarrayMatrix); bool _convertCvMat2ValarrayBuffer(InputArray inputMatToConvert, std::valarray<float> &outputValarrayMatrix);
//! private method called by constructors, gathers their parameters and use them in a unified way
void _init(const Size inputSize, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0);
}; };
// smart pointers allocation : // smart pointers allocation :
Ptr<Retina> createRetina(Size inputSize){ return new RetinaImpl(inputSize); } Ptr<Retina> createRetina(Size inputSize){ return new RetinaImpl(inputSize); }
Ptr<Retina> createRetina(Size inputSize, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod, const bool useRetinaLogSampling, const double reductionFactor, const double samplingStrenght){return new RetinaImpl(inputSize, colorMode, colorSamplingMethod, useRetinaLogSampling, reductionFactor, samplingStrenght);} Ptr<Retina> createRetina(Size inputSize, const bool colorMode, int colorSamplingMethod, const bool useRetinaLogSampling, const double reductionFactor, const double samplingStrenght){return new RetinaImpl(inputSize, colorMode, colorSamplingMethod, useRetinaLogSampling, reductionFactor, samplingStrenght);}
// RetinaImpl code // RetinaImpl code
RetinaImpl::RetinaImpl(const cv::Size inputSz) RetinaImpl::RetinaImpl(const cv::Size inputSz)
...@@ -294,7 +306,7 @@ RetinaImpl::RetinaImpl(const cv::Size inputSz) ...@@ -294,7 +306,7 @@ RetinaImpl::RetinaImpl(const cv::Size inputSz)
_init(inputSz, true, RETINA_COLOR_BAYER, false); _init(inputSz, true, RETINA_COLOR_BAYER, false);
} }
RetinaImpl::RetinaImpl(const cv::Size inputSz, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod, const bool useRetinaLogSampling, const double reductionFactor, const double samplingStrenght) RetinaImpl::RetinaImpl(const cv::Size inputSz, const bool colorMode, int colorSamplingMethod, const bool useRetinaLogSampling, const double reductionFactor, const double samplingStrenght)
{ {
_retinaFilter = 0; _retinaFilter = 0;
_init(inputSz, colorMode, colorSamplingMethod, useRetinaLogSampling, reductionFactor, samplingStrenght); _init(inputSz, colorMode, colorSamplingMethod, useRetinaLogSampling, reductionFactor, samplingStrenght);
...@@ -401,10 +413,10 @@ void RetinaImpl::setup(cv::FileStorage &fs, const bool applyDefaultSetupOnFailur ...@@ -401,10 +413,10 @@ void RetinaImpl::setup(cv::FileStorage &fs, const bool applyDefaultSetupOnFailur
printf("%s\n", printSetup().c_str()); printf("%s\n", printSetup().c_str());
} }
void RetinaImpl::setup(cv::Retina::RetinaParameters newConfiguration) void RetinaImpl::setup(Retina::RetinaParameters newConfiguration)
{ {
// simply copy structures // simply copy structures
memcpy(&_retinaParameters, &newConfiguration, sizeof(cv::Retina::RetinaParameters)); memcpy(&_retinaParameters, &newConfiguration, sizeof(Retina::RetinaParameters));
// apply setup // apply setup
setupOPLandIPLParvoChannel(_retinaParameters.OPLandIplParvo.colorMode, _retinaParameters.OPLandIplParvo.normaliseOutput, _retinaParameters.OPLandIplParvo.photoreceptorsLocalAdaptationSensitivity, _retinaParameters.OPLandIplParvo.photoreceptorsTemporalConstant, _retinaParameters.OPLandIplParvo.photoreceptorsSpatialConstant, _retinaParameters.OPLandIplParvo.horizontalCellsGain, _retinaParameters.OPLandIplParvo.hcellsTemporalConstant, _retinaParameters.OPLandIplParvo.hcellsSpatialConstant, _retinaParameters.OPLandIplParvo.ganglionCellsSensitivity); setupOPLandIPLParvoChannel(_retinaParameters.OPLandIplParvo.colorMode, _retinaParameters.OPLandIplParvo.normaliseOutput, _retinaParameters.OPLandIplParvo.photoreceptorsLocalAdaptationSensitivity, _retinaParameters.OPLandIplParvo.photoreceptorsTemporalConstant, _retinaParameters.OPLandIplParvo.photoreceptorsSpatialConstant, _retinaParameters.OPLandIplParvo.horizontalCellsGain, _retinaParameters.OPLandIplParvo.hcellsTemporalConstant, _retinaParameters.OPLandIplParvo.hcellsSpatialConstant, _retinaParameters.OPLandIplParvo.ganglionCellsSensitivity);
setupIPLMagnoChannel(_retinaParameters.IplMagno.normaliseOutput, _retinaParameters.IplMagno.parasolCells_beta, _retinaParameters.IplMagno.parasolCells_tau, _retinaParameters.IplMagno.parasolCells_k, _retinaParameters.IplMagno.amacrinCellsTemporalCutFrequency,_retinaParameters.IplMagno.V0CompressionParameter, _retinaParameters.IplMagno.localAdaptintegration_tau, _retinaParameters.IplMagno.localAdaptintegration_k); setupIPLMagnoChannel(_retinaParameters.IplMagno.normaliseOutput, _retinaParameters.IplMagno.parasolCells_beta, _retinaParameters.IplMagno.parasolCells_tau, _retinaParameters.IplMagno.parasolCells_k, _retinaParameters.IplMagno.amacrinCellsTemporalCutFrequency,_retinaParameters.IplMagno.V0CompressionParameter, _retinaParameters.IplMagno.localAdaptintegration_tau, _retinaParameters.IplMagno.localAdaptintegration_k);
...@@ -526,6 +538,27 @@ void RetinaImpl::run(InputArray inputMatToConvert) ...@@ -526,6 +538,27 @@ void RetinaImpl::run(InputArray inputMatToConvert)
throw cv::Exception(-1, "RetinaImpl cannot be applied, wrong input buffer size", "RetinaImpl::run", "RetinaImpl.h", 0); throw cv::Exception(-1, "RetinaImpl cannot be applied, wrong input buffer size", "RetinaImpl::run", "RetinaImpl.h", 0);
} }
void RetinaImpl::applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage)
{
// first convert input image to the compatible format :
const bool colorMode = _convertCvMat2ValarrayBuffer(inputImage.getMat(), _inputBuffer);
const unsigned int nbPixels=_retinaFilter->getOutputNBrows()*_retinaFilter->getOutputNBcolumns();
// process tone mapping
if (colorMode)
{
std::valarray<float> imageOutput(nbPixels*3);
_retinaFilter->runRGBToneMapping(_inputBuffer, imageOutput, true, _retinaParameters.OPLandIplParvo.photoreceptorsLocalAdaptationSensitivity, _retinaParameters.OPLandIplParvo.ganglionCellsSensitivity);
_convertValarrayBuffer2cvMat(imageOutput, _retinaFilter->getOutputNBrows(), _retinaFilter->getOutputNBcolumns(), true, outputToneMappedImage);
}else
{
std::valarray<float> imageOutput(nbPixels);
_retinaFilter->runGrayToneMapping(_inputBuffer, imageOutput, _retinaParameters.OPLandIplParvo.photoreceptorsLocalAdaptationSensitivity, _retinaParameters.OPLandIplParvo.ganglionCellsSensitivity);
_convertValarrayBuffer2cvMat(imageOutput, _retinaFilter->getOutputNBrows(), _retinaFilter->getOutputNBcolumns(), false, outputToneMappedImage);
}
}
void RetinaImpl::getParvo(OutputArray retinaOutput_parvo) void RetinaImpl::getParvo(OutputArray retinaOutput_parvo)
{ {
if (_retinaFilter->getColorMode()) if (_retinaFilter->getColorMode())
...@@ -580,11 +613,11 @@ const Mat RetinaImpl::getParvoRAW() const { ...@@ -580,11 +613,11 @@ const Mat RetinaImpl::getParvoRAW() const {
} }
// private method called by constructirs // private method called by constructirs
void RetinaImpl::_init(const cv::Size inputSz, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod, const bool useRetinaLogSampling, const double reductionFactor, const double samplingStrenght) void RetinaImpl::_init(const cv::Size inputSz, const bool colorMode, int colorSamplingMethod, const bool useRetinaLogSampling, const double reductionFactor, const double samplingStrenght)
{ {
// basic error check // basic error check
if (inputSz.height*inputSz.width <= 0) if (inputSz.height*inputSz.width <= 0)
throw cv::Exception(-1, "Bad retina size setup : size height and with must be superior to zero", "RetinaImpl::setup", "RetinaImpl.h", 0); throw cv::Exception(-1, "Bad retina size setup : size height and with must be superior to zero", "RetinaImpl::setup", "Retina.cpp", 0);
unsigned int nbPixels=inputSz.height*inputSz.width; unsigned int nbPixels=inputSz.height*inputSz.width;
// resize buffers if size does not match // resize buffers if size does not match
...@@ -623,7 +656,8 @@ void RetinaImpl::_convertValarrayBuffer2cvMat(const std::valarray<float> &grayMa ...@@ -623,7 +656,8 @@ void RetinaImpl::_convertValarrayBuffer2cvMat(const std::valarray<float> &grayMa
} }
}else }else
{ {
const unsigned int doubleNBpixels=_retinaFilter->getOutputNBpixels()*2; const unsigned int nbPixels=nbColumns*nbRows;
const unsigned int doubleNBpixels=nbColumns*nbRows*2;
outBuffer.create(cv::Size(nbColumns, nbRows), CV_8UC3); outBuffer.create(cv::Size(nbColumns, nbRows), CV_8UC3);
Mat outMat = outBuffer.getMat(); Mat outMat = outBuffer.getMat();
for (unsigned int i=0;i<nbRows;++i) for (unsigned int i=0;i<nbRows;++i)
...@@ -633,7 +667,7 @@ void RetinaImpl::_convertValarrayBuffer2cvMat(const std::valarray<float> &grayMa ...@@ -633,7 +667,7 @@ void RetinaImpl::_convertValarrayBuffer2cvMat(const std::valarray<float> &grayMa
cv::Point2d pixel(j,i); cv::Point2d pixel(j,i);
cv::Vec3b pixelValues; cv::Vec3b pixelValues;
pixelValues[2]=(unsigned char)*(valarrayPTR); pixelValues[2]=(unsigned char)*(valarrayPTR);
pixelValues[1]=(unsigned char)*(valarrayPTR+_retinaFilter->getOutputNBpixels()); pixelValues[1]=(unsigned char)*(valarrayPTR+nbPixels);
pixelValues[0]=(unsigned char)*(valarrayPTR+doubleNBpixels); pixelValues[0]=(unsigned char)*(valarrayPTR+doubleNBpixels);
outMat.at<cv::Vec3b>(pixel)=pixelValues; outMat.at<cv::Vec3b>(pixel)=pixelValues;
...@@ -656,15 +690,16 @@ bool RetinaImpl::_convertCvMat2ValarrayBuffer(InputArray inputMat, std::valarray ...@@ -656,15 +690,16 @@ bool RetinaImpl::_convertCvMat2ValarrayBuffer(InputArray inputMat, std::valarray
typedef float T; // define here the target pixel format, here, float typedef float T; // define here the target pixel format, here, float
const int dsttype = DataType<T>::depth; // output buffer is float format const int dsttype = DataType<T>::depth; // output buffer is float format
const unsigned int nbPixels=inputMat.getMat().rows*inputMat.getMat().cols;
const unsigned int doubleNBpixels=inputMat.getMat().rows*inputMat.getMat().cols*2;
if(imageNumberOfChannels==4) if(imageNumberOfChannels==4)
{ {
// create a cv::Mat table (for RGBA planes) // create a cv::Mat table (for RGBA planes)
cv::Mat planes[4] = cv::Mat planes[4] =
{ {
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[_retinaFilter->getInputNBpixels()*2]), cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[doubleNBpixels]),
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[_retinaFilter->getInputNBpixels()]), cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[nbPixels]),
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[0]) cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[0])
}; };
planes[3] = cv::Mat(inputMatToConvert.size(), dsttype); // last channel (alpha) does not point on the valarray (not usefull in our case) planes[3] = cv::Mat(inputMatToConvert.size(), dsttype); // last channel (alpha) does not point on the valarray (not usefull in our case)
...@@ -676,8 +711,8 @@ bool RetinaImpl::_convertCvMat2ValarrayBuffer(InputArray inputMat, std::valarray ...@@ -676,8 +711,8 @@ bool RetinaImpl::_convertCvMat2ValarrayBuffer(InputArray inputMat, std::valarray
// create a cv::Mat table (for RGB planes) // create a cv::Mat table (for RGB planes)
cv::Mat planes[] = cv::Mat planes[] =
{ {
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[_retinaFilter->getInputNBpixels()*2]), cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[doubleNBpixels]),
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[_retinaFilter->getInputNBpixels()]), cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[nbPixels]),
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[0]) cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[0])
}; };
// split color cv::Mat in 3 planes... it fills valarray directely // split color cv::Mat in 3 planes... it fills valarray directely
...@@ -701,5 +736,5 @@ void RetinaImpl::activateMovingContoursProcessing(const bool activate){_retinaFi ...@@ -701,5 +736,5 @@ void RetinaImpl::activateMovingContoursProcessing(const bool activate){_retinaFi
void RetinaImpl::activateContoursProcessing(const bool activate){_retinaFilter->activateContoursProcessing(activate);} void RetinaImpl::activateContoursProcessing(const bool activate){_retinaFilter->activateContoursProcessing(activate);}
} // end of namespace cv }// end of namespace bioinspired
}// end of namespace cv
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -73,14 +73,15 @@ ...@@ -73,14 +73,15 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
// init static values // init static values
static float _LMStoACr1Cr2[]={1.0, 1.0, 0.0, 1.0, -1.0, 0.0, -0.5, -0.5, 1.0}; static float _LMStoACr1Cr2[]={1.0, 1.0, 0.0, 1.0, -1.0, 0.0, -0.5, -0.5, 1.0};
//static double _ACr1Cr2toLMS[]={0.5, 0.5, 0.0, 0.5, -0.5, 0.0, 0.5, 0.0, 1.0}; //static double _ACr1Cr2toLMS[]={0.5, 0.5, 0.0, 0.5, -0.5, 0.0, 0.5, 0.0, 1.0};
static float _LMStoLab[]={0.5774f, 0.5774f, 0.5774f, 0.4082f, 0.4082f, -0.8165f, 0.7071f, -0.7071f, 0.f}; static float _LMStoLab[]={0.5774f, 0.5774f, 0.5774f, 0.4082f, 0.4082f, -0.8165f, 0.7071f, -0.7071f, 0.f};
// constructor/desctructor // constructor/desctructor
RetinaColor::RetinaColor(const unsigned int NBrows, const unsigned int NBcolumns, const RETINA_COLORSAMPLINGMETHOD samplingMethod) RetinaColor::RetinaColor(const unsigned int NBrows, const unsigned int NBcolumns, const int samplingMethod)
:BasicRetinaFilter(NBrows, NBcolumns, 3), :BasicRetinaFilter(NBrows, NBcolumns, 3),
_colorSampling(NBrows*NBcolumns), _colorSampling(NBrows*NBcolumns),
_RGBmosaic(NBrows*NBcolumns*3), _RGBmosaic(NBrows*NBcolumns*3),
...@@ -720,4 +721,5 @@ void RetinaColor::_applyImageColorSpaceConversion(const std::valarray<float> &in ...@@ -720,4 +721,5 @@ void RetinaColor::_applyImageColorSpaceConversion(const std::valarray<float> &in
} }
} }
} }// end of namespace bioinspired
}// end of namespace cv
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -85,7 +85,8 @@ ...@@ -85,7 +85,8 @@
namespace cv namespace cv
{ {
namespace bioinspired
{
class RetinaColor: public BasicRetinaFilter class RetinaColor: public BasicRetinaFilter
{ {
public: public:
...@@ -99,7 +100,7 @@ namespace cv ...@@ -99,7 +100,7 @@ namespace cv
* @param NBcolumns: number of columns of the input image * @param NBcolumns: number of columns of the input image
* @param samplingMethod: the chosen color sampling method * @param samplingMethod: the chosen color sampling method
*/ */
RetinaColor(const unsigned int NBrows, const unsigned int NBcolumns, const RETINA_COLORSAMPLINGMETHOD samplingMethod=RETINA_COLOR_DIAGONAL); RetinaColor(const unsigned int NBrows, const unsigned int NBcolumns, const int samplingMethod=RETINA_COLOR_BAYER);
/** /**
* standard destructor * standard destructor
...@@ -219,7 +220,7 @@ namespace cv ...@@ -219,7 +220,7 @@ namespace cv
protected: protected:
// private functions // private functions
RETINA_COLORSAMPLINGMETHOD _samplingMethod; int _samplingMethod;
bool _saturateColors; bool _saturateColors;
float _colorSaturationValue; float _colorSaturationValue;
// links to parent buffers (more convienient names // links to parent buffers (more convienient names
...@@ -382,8 +383,7 @@ namespace cv ...@@ -382,8 +383,7 @@ namespace cv
#endif #endif
}; };
} }// end of namespace bioinspired
}// end of namespace cv
#endif /*RETINACOLOR_HPP_*/ #endif /*RETINACOLOR_HPP_*/
/*#******************************************************************************
** IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
**
** By downloading, copying, installing or using the software you agree to this license.
** If you do not agree to this license, do not download, install,
** copy or use the software.
**
**
** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
**
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
**
** Creation - enhancement process 2007-2013
** Author: Alexandre Benoit (benoit.alexandre.vision@gmail.com), LISTIC lab, Annecy le vieux, France
**
** Theses algorithm have been developped by Alexandre BENOIT since his thesis with Alice Caplier at Gipsa-Lab (www.gipsa-lab.inpg.fr) and the research he pursues at LISTIC Lab (www.listic.univ-savoie.fr).
** Refer to the following research paper for more information:
** Benoit A., Caplier A., Durette B., Herault, J., "USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773, DOI: http://dx.doi.org/10.1016/j.cviu.2010.01.011
** This work have been carried out thanks to Jeanny Herault who's research and great discussions are the basis of all this work, please take a look at his book:
** Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891.
**
**
** This class is based on image processing tools of the author and already used within the Retina class (this is the same code as method retina::applyFastToneMapping, but in an independent class, it is ligth from a memory requirement point of view). It implements an adaptation of the efficient tone mapping algorithm propose by David Alleyson, Sabine Susstruck and Laurence Meylan's work, please cite:
** -> Meylan L., Alleysson D., and Susstrunk S., A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images, Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816
**
**
** License Agreement
** For Open Source Computer Vision Library
**
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
**
** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
**
** Third party copyrights are property of their respective owners.
**
** Redistribution and use in source and binary forms, with or without modification,
** are permitted provided that the following conditions are met:
**
** * Redistributions of source code must retain the above copyright notice,
** this list of conditions and the following disclaimer.
**
** * Redistributions in binary form must reproduce the above copyright notice,
** this list of conditions and the following disclaimer in the documentation
** and/or other materials provided with the distribution.
**
** * The name of the copyright holders may not be used to endorse or promote products
** derived from this software without specific prior written permission.
**
** This software is provided by the copyright holders and contributors "as is" and
** any express or implied warranties, including, but not limited to, the implied
** warranties of merchantability and fitness for a particular purpose are disclaimed.
** In no event shall the Intel Corporation or contributors be liable for any direct,
** indirect, incidental, special, exemplary, or consequential damages
** (including, but not limited to, procurement of substitute goods or services;
** loss of use, data, or profits; or business interruption) however caused
** and on any theory of liability, whether in contract, strict liability,
** or tort (including negligence or otherwise) arising in any way out of
** the use of this software, even if advised of the possibility of such damage.
*******************************************************************************/
/*
* retinafasttonemapping.cpp
*
* Created on: May 26, 2013
* Author: Alexandre Benoit
*/
#include "precomp.hpp"
#include "basicretinafilter.hpp"
#include "retinacolor.hpp"
#include <cstdio>
#include <sstream>
#include <valarray>
namespace cv
{
namespace bioinspired
{
/**
* @class RetinaFastToneMappingImpl a wrapper class which allows the tone mapping algorithm of Meylan&al(2007) to be used with OpenCV.
* This algorithm is already implemented in thre Retina class (retina::applyFastToneMapping) but used it does not require all the retina model to be allocated. This allows a light memory use for low memory devices (smartphones, etc.
* As a summary, these are the model properties:
* => 2 stages of local luminance adaptation with a different local neighborhood for each.
* => first stage models the retina photorecetors local luminance adaptation
* => second stage models th ganglion cells local information adaptation
* => compared to the initial publication, this class uses spatio-temporal low pass filters instead of spatial only filters.
* ====> this can help noise robustness and temporal stability for video sequence use cases.
* for more information, read to the following papers :
* Meylan L., Alleysson D., and Susstrunk S., A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images, Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816Benoit A., Caplier A., Durette B., Herault, J., "USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773, DOI: http://dx.doi.org/10.1016/j.cviu.2010.01.011
* regarding spatio-temporal filter and the bigger retina model :
* Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891.
*/
class RetinaFastToneMappingImpl : public RetinaFastToneMapping
{
public:
/**
* constructor
* @param imageInput: the size of the images to process
*/
RetinaFastToneMappingImpl(Size imageInput)
{
unsigned int nbPixels=imageInput.height*imageInput.width;
// basic error check
if (nbPixels <= 0)
throw cv::Exception(-1, "Bad retina size setup : size height and with must be superior to zero", "RetinaImpl::setup", "retinafasttonemapping.cpp", 0);
// resize buffers
_inputBuffer.resize(nbPixels*3); // buffer supports gray images but also 3 channels color buffers... (larger is better...)
_imageOutput.resize(nbPixels*3);
_temp2.resize(nbPixels);
// allocate the main filter with 2 setup sets properties (one for each low pass filter
_multiuseFilter = new BasicRetinaFilter(imageInput.height, imageInput.width, 2);
// allocate the color manager (multiplexer/demultiplexer
_colorEngine = new RetinaColor(imageInput.height, imageInput.width);
// setup filter behaviors with default values
setup();
}
/**
* basic destructor
*/
virtual ~RetinaFastToneMappingImpl(){};
/**
* method that applies a luminance correction (initially High Dynamic Range (HDR) tone mapping) using only the 2 local adaptation stages of the retina parvocellular channel : photoreceptors level and ganlion cells level. Spatio temporal filtering is applied but limited to temporal smoothing and eventually high frequencies attenuation. This is a lighter method than the one available using the regular retina::run method. It is then faster but it does not include complete temporal filtering nor retina spectral whitening. Then, it can have a more limited effect on images with a very high dynamic range. This is an adptation of the original still image HDR tone mapping algorithm of David Alleyson, Sabine Susstruck and Laurence Meylan's work, please cite:
* -> Meylan L., Alleysson D., and Susstrunk S., A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images, Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816
@param inputImage the input image to process RGB or gray levels
@param outputToneMappedImage the output tone mapped image
*/
virtual void applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage)
{
// first convert input image to the compatible format :
const bool colorMode = _convertCvMat2ValarrayBuffer(inputImage.getMat(), _inputBuffer);
// process tone mapping
if (colorMode)
{
_runRGBToneMapping(_inputBuffer, _imageOutput, true);
_convertValarrayBuffer2cvMat(_imageOutput, _multiuseFilter->getNBrows(), _multiuseFilter->getNBcolumns(), true, outputToneMappedImage);
}else
{
_runGrayToneMapping(_inputBuffer, _imageOutput);
_convertValarrayBuffer2cvMat(_imageOutput, _multiuseFilter->getNBrows(), _multiuseFilter->getNBcolumns(), false, outputToneMappedImage);
}
}
/**
* setup method that updates tone mapping behaviors by adjusing the local luminance computation area
* @param photoreceptorsNeighborhoodRadius the first stage local adaptation area
* @param ganglioncellsNeighborhoodRadius the second stage local adaptation area
* @param meanLuminanceModulatorK the factor applied to modulate the meanLuminance information (default is 1, see reference paper)
*/
virtual void setup(const float photoreceptorsNeighborhoodRadius=3.f, const float ganglioncellsNeighborhoodRadius=1.f, const float meanLuminanceModulatorK=1.f)
{
// setup the spatio-temporal properties of each filter
_meanLuminanceModulatorK = meanLuminanceModulatorK;
_multiuseFilter->setV0CompressionParameter(1.f, 255.f, 128.f);
_multiuseFilter->setLPfilterParameters(0.f, 0.f, photoreceptorsNeighborhoodRadius, 1);
_multiuseFilter->setLPfilterParameters(0.f, 0.f, ganglioncellsNeighborhoodRadius, 2);
}
private:
// a filter able to perform local adaptation and low pass spatio-temporal filtering
cv::Ptr <BasicRetinaFilter> _multiuseFilter;
cv::Ptr <RetinaColor> _colorEngine;
//!< buffer used to convert input cv::Mat to internal retina buffers format (valarrays)
std::valarray<float> _inputBuffer;
std::valarray<float> _imageOutput;
std::valarray<float> _temp2;
float _meanLuminanceModulatorK;
void _convertValarrayBuffer2cvMat(const std::valarray<float> &grayMatrixToConvert, const unsigned int nbRows, const unsigned int nbColumns, const bool colorMode, OutputArray outBuffer)
{
// fill output buffer with the valarray buffer
const float *valarrayPTR=get_data(grayMatrixToConvert);
if (!colorMode)
{
outBuffer.create(cv::Size(nbColumns, nbRows), CV_8U);
Mat outMat = outBuffer.getMat();
for (unsigned int i=0;i<nbRows;++i)
{
for (unsigned int j=0;j<nbColumns;++j)
{
cv::Point2d pixel(j,i);
outMat.at<unsigned char>(pixel)=(unsigned char)*(valarrayPTR++);
}
}
}else
{
const unsigned int nbPixels=nbColumns*nbRows;
const unsigned int doubleNBpixels=nbColumns*nbRows*2;
outBuffer.create(cv::Size(nbColumns, nbRows), CV_8UC3);
Mat outMat = outBuffer.getMat();
for (unsigned int i=0;i<nbRows;++i)
{
for (unsigned int j=0;j<nbColumns;++j,++valarrayPTR)
{
cv::Point2d pixel(j,i);
cv::Vec3b pixelValues;
pixelValues[2]=(unsigned char)*(valarrayPTR);
pixelValues[1]=(unsigned char)*(valarrayPTR+nbPixels);
pixelValues[0]=(unsigned char)*(valarrayPTR+doubleNBpixels);
outMat.at<cv::Vec3b>(pixel)=pixelValues;
}
}
}
}
bool _convertCvMat2ValarrayBuffer(InputArray inputMat, std::valarray<float> &outputValarrayMatrix)
{
const Mat inputMatToConvert=inputMat.getMat();
// first check input consistency
if (inputMatToConvert.empty())
throw cv::Exception(-1, "RetinaImpl cannot be applied, input buffer is empty", "RetinaImpl::run", "RetinaImpl.h", 0);
// retreive color mode from image input
int imageNumberOfChannels = inputMatToConvert.channels();
// convert to float AND fill the valarray buffer
typedef float T; // define here the target pixel format, here, float
const int dsttype = DataType<T>::depth; // output buffer is float format
const unsigned int nbPixels=inputMat.getMat().rows*inputMat.getMat().cols;
const unsigned int doubleNBpixels=inputMat.getMat().rows*inputMat.getMat().cols*2;
if(imageNumberOfChannels==4)
{
// create a cv::Mat table (for RGBA planes)
cv::Mat planes[4] =
{
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[doubleNBpixels]),
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[nbPixels]),
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[0])
};
planes[3] = cv::Mat(inputMatToConvert.size(), dsttype); // last channel (alpha) does not point on the valarray (not usefull in our case)
// split color cv::Mat in 4 planes... it fills valarray directely
cv::split(Mat_<Vec<T, 4> >(inputMatToConvert), planes);
}
else if (imageNumberOfChannels==3)
{
// create a cv::Mat table (for RGB planes)
cv::Mat planes[] =
{
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[doubleNBpixels]),
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[nbPixels]),
cv::Mat(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[0])
};
// split color cv::Mat in 3 planes... it fills valarray directely
cv::split(cv::Mat_<Vec<T, 3> >(inputMatToConvert), planes);
}
else if(imageNumberOfChannels==1)
{
// create a cv::Mat header for the valarray
cv::Mat dst(inputMatToConvert.size(), dsttype, &outputValarrayMatrix[0]);
inputMatToConvert.convertTo(dst, dsttype);
}
else
CV_Error(Error::StsUnsupportedFormat, "input image must be single channel (gray levels), bgr format (color) or bgra (color with transparency which won't be considered");
return imageNumberOfChannels>1; // return bool : false for gray level image processing, true for color mode
}
// run the initilized retina filter in order to perform gray image tone mapping, after this call all retina outputs are updated
void _runGrayToneMapping(const std::valarray<float> &grayImageInput, std::valarray<float> &grayImageOutput)
{
// apply tone mapping on the multiplexed image
// -> photoreceptors local adaptation (large area adaptation)
_multiuseFilter->runFilter_LPfilter(grayImageInput, grayImageOutput, 0); // compute low pass filtering modeling the horizontal cells filtering to acess local luminance
_multiuseFilter->setV0CompressionParameterToneMapping(1.f, grayImageOutput.max(), _meanLuminanceModulatorK*grayImageOutput.sum()/(float)_multiuseFilter->getNBpixels());
_multiuseFilter->runFilter_LocalAdapdation(grayImageInput, grayImageOutput, _temp2); // adapt contrast to local luminance
// -> ganglion cells local adaptation (short area adaptation)
_multiuseFilter->runFilter_LPfilter(_temp2, grayImageOutput, 1); // compute low pass filtering (high cut frequency (remove spatio-temporal noise)
_multiuseFilter->setV0CompressionParameterToneMapping(1.f, _temp2.max(), _meanLuminanceModulatorK*grayImageOutput.sum()/(float)_multiuseFilter->getNBpixels());
_multiuseFilter->runFilter_LocalAdapdation(_temp2, grayImageOutput, grayImageOutput); // adapt contrast to local luminance
}
// run the initilized retina filter in order to perform color tone mapping, after this call all retina outputs are updated
void _runRGBToneMapping(const std::valarray<float> &RGBimageInput, std::valarray<float> &RGBimageOutput, const bool useAdaptiveFiltering)
{
// multiplex the image with the color sampling method specified in the constructor
_colorEngine->runColorMultiplexing(RGBimageInput);
// apply tone mapping on the multiplexed image
_runGrayToneMapping(_colorEngine->getMultiplexedFrame(), RGBimageOutput);
// demultiplex tone maped image
_colorEngine->runColorDemultiplexing(RGBimageOutput, useAdaptiveFiltering, _multiuseFilter->getMaxInputValue());//_ColorEngine->getMultiplexedFrame());//_ParvoRetinaFilter->getPhotoreceptorsLPfilteringOutput());
// rescaling result between 0 and 255
_colorEngine->normalizeRGBOutput_0_maxOutputValue(255.0);
// return the result
RGBimageOutput=_colorEngine->getDemultiplexedColorFrame();
}
};
CV_EXPORTS Ptr<RetinaFastToneMapping> createRetinaFastToneMapping(Size inputSize)
{
return new RetinaFastToneMappingImpl(inputSize);
}
}// end of namespace bioinspired
}// end of namespace cv
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -72,9 +72,11 @@ ...@@ -72,9 +72,11 @@
#include <cmath> #include <cmath>
namespace cv namespace cv
{
namespace bioinspired
{ {
// standard constructor without any log sampling of the input frame // standard constructor without any log sampling of the input frame
RetinaFilter::RetinaFilter(const unsigned int sizeRows, const unsigned int sizeColumns, const bool colorMode, const RETINA_COLORSAMPLINGMETHOD samplingMethod, const bool useRetinaLogSampling, const double reductionFactor, const double samplingStrenght) RetinaFilter::RetinaFilter(const unsigned int sizeRows, const unsigned int sizeColumns, const bool colorMode, const int samplingMethod, const bool useRetinaLogSampling, const double reductionFactor, const double samplingStrenght)
: :
_retinaParvoMagnoMappedFrame(0), _retinaParvoMagnoMappedFrame(0),
_retinaParvoMagnoMapCoefTable(0), _retinaParvoMagnoMapCoefTable(0),
...@@ -375,21 +377,15 @@ namespace cv ...@@ -375,21 +377,15 @@ namespace cv
// apply tone mapping on the multiplexed image // apply tone mapping on the multiplexed image
// -> photoreceptors local adaptation (large area adaptation) // -> photoreceptors local adaptation (large area adaptation)
_photoreceptorsPrefilter.runFilter_LPfilter(grayImageInput, grayImageOutput, 2); // compute low pass filtering modeling the horizontal cells filtering to acess local luminance _photoreceptorsPrefilter.runFilter_LPfilter(grayImageInput, grayImageOutput, 2); // compute low pass filtering modeling the horizontal cells filtering to acess local luminance
_photoreceptorsPrefilter.setV0CompressionParameterToneMapping(PhotoreceptorsCompression, grayImageOutput.sum()/(float)_photoreceptorsPrefilter.getNBpixels()); _photoreceptorsPrefilter.setV0CompressionParameterToneMapping(1.f-PhotoreceptorsCompression, grayImageOutput.max(), 1.f*grayImageOutput.sum()/(float)_photoreceptorsPrefilter.getNBpixels());
_photoreceptorsPrefilter.runFilter_LocalAdapdation(grayImageInput, grayImageOutput, temp2); // adapt contrast to local luminance _photoreceptorsPrefilter.runFilter_LocalAdapdation(grayImageInput, grayImageOutput, temp2); // adapt contrast to local luminance
// high pass filter
//_spatiotemporalLPfilter(_localBuffer, _filterOutput, 2); // compute low pass filtering (high cut frequency (remove spatio-temporal noise)
//for (unsigned int i=0;i<_NBpixels;++i)
// _localBuffer[i]-= _filterOutput[i]/2.0;
// -> ganglion cells local adaptation (short area adaptation) // -> ganglion cells local adaptation (short area adaptation)
_photoreceptorsPrefilter.runFilter_LPfilter(temp2, grayImageOutput, 1); // compute low pass filtering (high cut frequency (remove spatio-temporal noise) _photoreceptorsPrefilter.runFilter_LPfilter(temp2, grayImageOutput, 1); // compute low pass filtering (high cut frequency (remove spatio-temporal noise)
_photoreceptorsPrefilter.setV0CompressionParameterToneMapping(ganglionCellsCompression, temp2.max(), temp2.sum()/(float)_photoreceptorsPrefilter.getNBpixels()); _photoreceptorsPrefilter.setV0CompressionParameterToneMapping(1.f-ganglionCellsCompression, temp2.max(), 1.f*temp2.sum()/(float)_photoreceptorsPrefilter.getNBpixels());
_photoreceptorsPrefilter.runFilter_LocalAdapdation(temp2, grayImageOutput, grayImageOutput); // adapt contrast to local luminance _photoreceptorsPrefilter.runFilter_LocalAdapdation(temp2, grayImageOutput, grayImageOutput); // adapt contrast to local luminance
} }
// run the initilized retina filter in order to perform color tone mapping, after this call all retina outputs are updated // run the initilized retina filter in order to perform color tone mapping, after this call all retina outputs are updated
void RetinaFilter::runRGBToneMapping(const std::valarray<float> &RGBimageInput, std::valarray<float> &RGBimageOutput, const bool useAdaptiveFiltering, const float PhotoreceptorsCompression, const float ganglionCellsCompression) void RetinaFilter::runRGBToneMapping(const std::valarray<float> &RGBimageInput, std::valarray<float> &RGBimageOutput, const bool useAdaptiveFiltering, const float PhotoreceptorsCompression, const float ganglionCellsCompression)
{ {
...@@ -526,4 +522,5 @@ namespace cv ...@@ -526,4 +522,5 @@ namespace cv
return true; return true;
} }
} }// end of namespace bioinspired
}// end of namespace cv
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -110,7 +110,8 @@ ...@@ -110,7 +110,8 @@
//#define __RETINADEBUG // define RETINADEBUG to display debug data //#define __RETINADEBUG // define RETINADEBUG to display debug data
namespace cv namespace cv
{ {
namespace bioinspired
{
// retina class that process the 3 outputs of the retina filtering stages // retina class that process the 3 outputs of the retina filtering stages
class RetinaFilter//: public BasicRetinaFilter class RetinaFilter//: public BasicRetinaFilter
{ {
...@@ -126,7 +127,7 @@ public: ...@@ -126,7 +127,7 @@ public:
* @param reductionFactor: only usefull if param useRetinaLogSampling=true, specifies the reduction factor of the output frame (as the center (fovea) is high resolution and corners can be underscaled, then a reduction of the output is allowed without precision leak * @param reductionFactor: only usefull if param useRetinaLogSampling=true, specifies the reduction factor of the output frame (as the center (fovea) is high resolution and corners can be underscaled, then a reduction of the output is allowed without precision leak
* @param samplingStrenght: only usefull if param useRetinaLogSampling=true, specifies the strenght of the log scale that is applied * @param samplingStrenght: only usefull if param useRetinaLogSampling=true, specifies the strenght of the log scale that is applied
*/ */
RetinaFilter(const unsigned int sizeRows, const unsigned int sizeColumns, const bool colorMode=false, const RETINA_COLORSAMPLINGMETHOD samplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0); RetinaFilter(const unsigned int sizeRows, const unsigned int sizeColumns, const bool colorMode=false, const int samplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0);
/** /**
* standard destructor * standard destructor
...@@ -541,9 +542,7 @@ private: ...@@ -541,9 +542,7 @@ private:
}; };
} }// end of namespace bioinspired
#endif /*RETINACLASSES_H_*/ }// end of namespace cv
#endif /*RETINACLASSES_H_*/
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
** copy or use the software. ** copy or use the software.
** **
** **
** HVStools : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab.
** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping.
** **
** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications)
...@@ -32,7 +32,7 @@ ...@@ -32,7 +32,7 @@
** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved.
** **
** For Human Visual System tools (hvstools) ** For Human Visual System tools (bioinspired)
** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved.
** **
** Third party copyrights are property of their respective owners. ** Third party copyrights are property of their respective owners.
...@@ -71,6 +71,12 @@ ...@@ -71,6 +71,12 @@
#include <cmath> #include <cmath>
//#define __TEMPLATEBUFFERDEBUG //define TEMPLATEBUFFERDEBUG in order to display debug information
namespace cv
{
namespace bioinspired
{
//// If a parallelization method is available then, you should define MAKE_PARALLEL, in the other case, the classical serial code will be used //// If a parallelization method is available then, you should define MAKE_PARALLEL, in the other case, the classical serial code will be used
#define MAKE_PARALLEL #define MAKE_PARALLEL
// ==> then include required includes // ==> then include required includes
...@@ -101,10 +107,6 @@ public: ...@@ -101,10 +107,6 @@ public:
}; };
#endif #endif
//#define __TEMPLATEBUFFERDEBUG //define TEMPLATEBUFFERDEBUG in order to display debug information
namespace cv
{
/** /**
* @class TemplateBuffer * @class TemplateBuffer
* @brief this class is a simple template memory buffer which contains basic functions to get information on or normalize the buffer content * @brief this class is a simple template memory buffer which contains basic functions to get information on or normalize the buffer content
...@@ -548,8 +550,6 @@ namespace cv ...@@ -548,8 +550,6 @@ namespace cv
return std::fabs(x); return std::fabs(x);
} }
} }// end of namespace bioinspired
}// end of namespace cv
#endif #endif
#include "test_precomp.hpp"
CV_TEST_MAIN("cv")
#ifdef __GNUC__
# pragma GCC diagnostic ignored "-Wmissing-declarations"
# if defined __clang__ || defined __APPLE__
# pragma GCC diagnostic ignored "-Wmissing-prototypes"
# pragma GCC diagnostic ignored "-Wextra"
# endif
#endif
#ifndef __OPENCV_TEST_PRECOMP_HPP__
#define __OPENCV_TEST_PRECOMP_HPP__
#include "opencv2/ts.hpp"
#include "opencv2/bioinspired.hpp"
#include <iostream>
#endif
...@@ -9,5 +9,4 @@ The module contains some recently added functionality that has not been stabiliz ...@@ -9,5 +9,4 @@ The module contains some recently added functionality that has not been stabiliz
stereo stereo
FaceRecognizer Documentation <facerec/index> FaceRecognizer Documentation <facerec/index>
Retina Documentation <retina/index>
openfabmap openfabmap
...@@ -633,7 +633,6 @@ CV_EXPORTS_W void applyColorMap(InputArray src, OutputArray dst, int colormap); ...@@ -633,7 +633,6 @@ CV_EXPORTS_W void applyColorMap(InputArray src, OutputArray dst, int colormap);
CV_EXPORTS bool initModule_contrib(); CV_EXPORTS bool initModule_contrib();
} }
#include "opencv2/contrib/retina.hpp"
#include "opencv2/contrib/openfabmap.hpp" #include "opencv2/contrib/openfabmap.hpp"
#endif #endif
...@@ -5,7 +5,7 @@ ...@@ -5,7 +5,7 @@
SET(OPENCV_CPP_SAMPLES_REQUIRED_DEPS opencv_core opencv_flann opencv_imgproc SET(OPENCV_CPP_SAMPLES_REQUIRED_DEPS opencv_core opencv_flann opencv_imgproc
opencv_highgui opencv_ml opencv_video opencv_objdetect opencv_photo opencv_nonfree opencv_softcascade opencv_highgui opencv_ml opencv_video opencv_objdetect opencv_photo opencv_nonfree opencv_softcascade
opencv_features2d opencv_calib3d opencv_legacy opencv_contrib opencv_stitching opencv_videostab) opencv_features2d opencv_calib3d opencv_legacy opencv_contrib opencv_stitching opencv_videostab opencv_bioinspired)
ocv_check_dependencies(${OPENCV_CPP_SAMPLES_REQUIRED_DEPS}) ocv_check_dependencies(${OPENCV_CPP_SAMPLES_REQUIRED_DEPS})
......
...@@ -10,8 +10,9 @@ ...@@ -10,8 +10,9 @@
#include <iostream> #include <iostream>
#include <cstring> #include <cstring>
#include "opencv2/contrib.hpp" #include "opencv2/bioinspired.hpp" // retina based algorithms
#include "opencv2/highgui.hpp" #include "opencv2/imgproc.hpp" // cvCvtcolor function
#include "opencv2/highgui.hpp" // display
static void help(std::string errorMessage) static void help(std::string errorMessage)
{ {
...@@ -127,7 +128,7 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i ...@@ -127,7 +128,7 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
normalize(imageInputRescaled, imageInputRescaled, 0.0, 255.0, cv::NORM_MINMAX); normalize(imageInputRescaled, imageInputRescaled, 0.0, 255.0, cv::NORM_MINMAX);
} }
cv::Ptr<cv::Retina> retina; cv::Ptr<cv::bioinspired::Retina> retina;
int retinaHcellsGain; int retinaHcellsGain;
int localAdaptation_photoreceptors, localAdaptation_Gcells; int localAdaptation_photoreceptors, localAdaptation_Gcells;
static void callBack_updateRetinaParams(int, void*) static void callBack_updateRetinaParams(int, void*)
...@@ -175,6 +176,12 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i ...@@ -175,6 +176,12 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
} }
bool useLogSampling = !strcmp(argv[argc-1], "log"); // check if user wants retina log sampling processing bool useLogSampling = !strcmp(argv[argc-1], "log"); // check if user wants retina log sampling processing
int chosenMethod=0;
if (!strcmp(argv[argc-1], "fast"))
{
chosenMethod=1;
std::cout<<"Using fast method (no spectral whithning), adaptation of Meylan&al 2008 method"<<std::endl;
}
std::string inputImageName=argv[1]; std::string inputImageName=argv[1];
...@@ -211,10 +218,15 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i ...@@ -211,10 +218,15 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
*/ */
if (useLogSampling) if (useLogSampling)
{ {
retina = cv::createRetina(inputImage.size(),true, cv::RETINA_COLOR_BAYER, true, 2.0, 10.0); retina = cv::bioinspired::createRetina(inputImage.size(),true, cv::bioinspired::RETINA_COLOR_BAYER, true, 2.0, 10.0);
} }
else// -> else allocate "classical" retina : else// -> else allocate "classical" retina :
retina = cv::createRetina(inputImage.size()); retina = cv::bioinspired::createRetina(inputImage.size());
// create a fast retina tone mapper (Meyla&al algorithm)
std::cout<<"Allocating fast tone mapper..."<<std::endl;
//cv::Ptr<cv::RetinaFastToneMapping> fastToneMapper=createRetinaFastToneMapping(inputImage.size());
std::cout<<"Fast tone mapper allocated"<<std::endl;
// save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup" // save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup"
retina->write("RetinaDefaultParameters.xml"); retina->write("RetinaDefaultParameters.xml");
...@@ -230,20 +242,19 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i ...@@ -230,20 +242,19 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
histogramClippingValue=0; // default value... updated with interface slider histogramClippingValue=0; // default value... updated with interface slider
//inputRescaleMat = inputImage; //inputRescaleMat = inputImage;
//outputRescaleMat = imageInputRescaled; //outputRescaleMat = imageInputRescaled;
cv::namedWindow("Retina input image (with cut edges histogram for basic pixels error avoidance)",1); cv::namedWindow("Processing configuration",1);
cv::createTrackbar("histogram edges clipping limit", "Retina input image (with cut edges histogram for basic pixels error avoidance)",&histogramClippingValue,50,callBack_rescaleGrayLevelMat); cv::createTrackbar("histogram edges clipping limit", "Processing configuration",&histogramClippingValue,50,callBack_rescaleGrayLevelMat);
cv::namedWindow("Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", 1);
colorSaturationFactor=3; colorSaturationFactor=3;
cv::createTrackbar("Color saturation", "Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", &colorSaturationFactor,5,callback_saturateColors); cv::createTrackbar("Color saturation", "Processing configuration", &colorSaturationFactor,5,callback_saturateColors);
retinaHcellsGain=40; retinaHcellsGain=40;
cv::createTrackbar("Hcells gain", "Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping",&retinaHcellsGain,100,callBack_updateRetinaParams); cv::createTrackbar("Hcells gain", "Processing configuration",&retinaHcellsGain,100,callBack_updateRetinaParams);
localAdaptation_photoreceptors=197; localAdaptation_photoreceptors=197;
localAdaptation_Gcells=190; localAdaptation_Gcells=190;
cv::createTrackbar("Ph sensitivity", "Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", &localAdaptation_photoreceptors,199,callBack_updateRetinaParams); cv::createTrackbar("Ph sensitivity", "Processing configuration", &localAdaptation_photoreceptors,199,callBack_updateRetinaParams);
cv::createTrackbar("Gcells sensitivity", "Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", &localAdaptation_Gcells,199,callBack_updateRetinaParams); cv::createTrackbar("Gcells sensitivity", "Processing configuration", &localAdaptation_Gcells,199,callBack_updateRetinaParams);
///////////////////////////////////////////// /////////////////////////////////////////////
...@@ -257,11 +268,28 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i ...@@ -257,11 +268,28 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
while(continueProcessing) while(continueProcessing)
{ {
// run retina filter // run retina filter
if (!chosenMethod)
{
retina->run(imageInputRescaled); retina->run(imageInputRescaled);
// Retrieve and display retina output // Retrieve and display retina output
retina->getParvo(retinaOutput_parvo); retina->getParvo(retinaOutput_parvo);
cv::imshow("Retina input image (with cut edges histogram for basic pixels error avoidance)", imageInputRescaled/255.0); cv::imshow("Retina input image (with cut edges histogram for basic pixels error avoidance)", imageInputRescaled/255.0);
cv::imshow("Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", retinaOutput_parvo); cv::imshow("Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", retinaOutput_parvo);
cv::imwrite("HDRinput.jpg",imageInputRescaled/255.0);
cv::imwrite("RetinaToneMapping.jpg",retinaOutput_parvo);
}
else
{
// apply the simplified hdr tone mapping method
cv::Mat fastToneMappingOutput;
retina->applyFastToneMapping(imageInputRescaled, fastToneMappingOutput);
cv::imshow("Retina fast tone mapping output : 16bit=>8bit image retina tonemapping", fastToneMappingOutput);
}
/*cv::Mat fastToneMappingOutput_specificObject;
fastToneMapper->setup(3.f, 1.5f, 1.f);
fastToneMapper->applyFastToneMapping(imageInputRescaled, fastToneMappingOutput_specificObject);
cv::imshow("### Retina fast tone mapping output : 16bit=>8bit image retina tonemapping", fastToneMappingOutput_specificObject);
*/
cv::waitKey(10); cv::waitKey(10);
} }
}catch(cv::Exception e) }catch(cv::Exception e)
......
...@@ -14,8 +14,9 @@ ...@@ -14,8 +14,9 @@
#include <stdio.h> #include <stdio.h>
#include <cstring> #include <cstring>
#include "opencv2/contrib.hpp" #include "opencv2/bioinspired.hpp" // retina based algorithms
#include "opencv2/highgui.hpp" #include "opencv2/imgproc.hpp" // cvCvtcolor function
#include "opencv2/highgui.hpp" // display
static void help(std::string errorMessage) static void help(std::string errorMessage)
{ {
...@@ -160,7 +161,7 @@ static void rescaleGrayLevelMat(const cv::Mat &inputMat, cv::Mat &outputMat, con ...@@ -160,7 +161,7 @@ static void rescaleGrayLevelMat(const cv::Mat &inputMat, cv::Mat &outputMat, con
} }
cv::Ptr<cv::Retina> retina; cv::Ptr<cv::bioinspired::Retina> retina;
int retinaHcellsGain; int retinaHcellsGain;
int localAdaptation_photoreceptors, localAdaptation_Gcells; int localAdaptation_photoreceptors, localAdaptation_Gcells;
static void callBack_updateRetinaParams(int, void*) static void callBack_updateRetinaParams(int, void*)
...@@ -280,10 +281,10 @@ static void loadNewFrame(const std::string filenamePrototype, const int currentF ...@@ -280,10 +281,10 @@ static void loadNewFrame(const std::string filenamePrototype, const int currentF
*/ */
if (useLogSampling) if (useLogSampling)
{ {
retina = cv::createRetina(inputImage.size(),true, cv::RETINA_COLOR_BAYER, true, 2.0, 10.0); retina = cv::bioinspired::createRetina(inputImage.size(),true, cv::bioinspired::RETINA_COLOR_BAYER, true, 2.0, 10.0);
} }
else// -> else allocate "classical" retina : else// -> else allocate "classical" retina :
retina = cv::createRetina(inputImage.size()); retina = cv::bioinspired::createRetina(inputImage.size());
// save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup" // save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup"
retina->write("RetinaDefaultParameters.xml"); retina->write("RetinaDefaultParameters.xml");
......
...@@ -9,7 +9,7 @@ ...@@ -9,7 +9,7 @@
#include <iostream> #include <iostream>
#include <cstring> #include <cstring>
#include "opencv2/contrib.hpp" #include "opencv2/bioinspired.hpp"
#include "opencv2/highgui.hpp" #include "opencv2/highgui.hpp"
static void help(std::string errorMessage) static void help(std::string errorMessage)
...@@ -106,15 +106,15 @@ int main(int argc, char* argv[]) { ...@@ -106,15 +106,15 @@ int main(int argc, char* argv[]) {
try try
{ {
// create a retina instance with default parameters setup, uncomment the initialisation you wanna test // create a retina instance with default parameters setup, uncomment the initialisation you wanna test
cv::Ptr<cv::Retina> myRetina; cv::Ptr<cv::bioinspired::Retina> myRetina;
// if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision) // if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision)
if (useLogSampling) if (useLogSampling)
{ {
myRetina = cv::createRetina(inputFrame.size(), true, cv::RETINA_COLOR_BAYER, true, 2.0, 10.0); myRetina = cv::bioinspired::createRetina(inputFrame.size(), true, cv::bioinspired::RETINA_COLOR_BAYER, true, 2.0, 10.0);
} }
else// -> else allocate "classical" retina : else// -> else allocate "classical" retina :
myRetina = cv::createRetina(inputFrame.size()); myRetina = cv::bioinspired::createRetina(inputFrame.size());
// save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup" // save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup"
myRetina->write("RetinaDefaultParameters.xml"); myRetina->write("RetinaDefaultParameters.xml");
...@@ -143,7 +143,8 @@ int main(int argc, char* argv[]) { ...@@ -143,7 +143,8 @@ int main(int argc, char* argv[]) {
cv::imshow("retina input", inputFrame); cv::imshow("retina input", inputFrame);
cv::imshow("Retina Parvo", retinaOutput_parvo); cv::imshow("Retina Parvo", retinaOutput_parvo);
cv::imshow("Retina Magno", retinaOutput_magno); cv::imshow("Retina Magno", retinaOutput_magno);
cv::waitKey(10);
cv::waitKey(5);
} }
}catch(cv::Exception e) }catch(cv::Exception e)
{ {
......
...@@ -9,7 +9,7 @@ ...@@ -9,7 +9,7 @@
#include <iostream> #include <iostream>
#include <cstring> #include <cstring>
#include "opencv2/contrib.hpp" #include "opencv2/bioinspired.hpp"
#include "opencv2/highgui.hpp" #include "opencv2/highgui.hpp"
static void help(std::string errorMessage) static void help(std::string errorMessage)
...@@ -95,16 +95,16 @@ int main(int argc, char* argv[]) { ...@@ -95,16 +95,16 @@ int main(int argc, char* argv[]) {
try try
{ {
// create a retina instance with default parameters setup, uncomment the initialisation you wanna test // create a retina instance with default parameters setup, uncomment the initialisation you wanna test
cv::Ptr<cv::Retina> myRetina; cv::Ptr<cv::bioinspired::Retina> myRetina;
// if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision) // if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision)
if (useLogSampling) if (useLogSampling)
{ {
myRetina = cv::createRetina(inputFrame.size(), true, cv::RETINA_COLOR_BAYER, true, 2.0, 10.0); myRetina = cv::bioinspired::createRetina(inputFrame.size(), true, cv::bioinspired::RETINA_COLOR_BAYER, true, 2.0, 10.0);
} }
else// -> else allocate "classical" retina : else// -> else allocate "classical" retina :
{ {
myRetina = cv::createRetina(inputFrame.size()); myRetina = cv::bioinspired::createRetina(inputFrame.size());
} }
// save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup" // save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup"
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册