Opencv for PD

these squares, and the differences between them.

Using this technique, we can detect some peculiar form in an image ( face, mouth, eyes, ... ), but this needs to be trained to have the right coefficients stored in the XML with

a great number of sample images ( > 10.000 ). Open CV provides a tool to create such a decision tree, but the task is fastidious and not easy. 

There are various haar's cascade available on the web like this collection : [[1]].

Input Parameters :

  load <xml file> : load an XML Haar's cascade decision tree.

  mode (0/1) : use edge detection or not.

  min_size : minimal size of a detected object.

  min_neighbours : minimum number (minus 1) of neighbour rectangles that makes up an object ( default 2 ).

  scale_factor : scale applied for block size.

  ftolerance : number of frames where an object can temporarily disappear ( used for identification / numbering ).

  clear : clear all objects markers.

Output :

  second outlet : number of detected objects.

  third outlet : coordinates of each object identified by a circle under the form 'number x y radius'.

Template:Start:pdp opencv haarcascade.png

pdp_opencv_camshift / pix_opencv_camshift

This object implements the Continuously Adaptive Mean-shift tracking algorithm (CAM).

You can select an object with the mouse and it starts tracking the object from a rectangular zone around this point, but adapting the zone at each iteration.

It gives also the orientation of the object. It's working in the HSV colorspace, distinguishing the hue from other fields S, V.

Input Parameters :

  vmin : V pre-filtering minimal value.

  vmax : V pre-filtering maximal value.

  smin : S pre-filtering minimal value.

  backproject (0/1) : show the backproject ( binary image of selected pixels ).

  rwidth : initial width of the tracking zone.

  rheight : initial height of the tracking zone.

  track x y : track an object containing this point.

Output :

  second outlet : coordinates, size and orientation of the track object ( only one ) in the form : 'x y width height angle'.

Template:Start:pdp opencv camshift.png

pdp_opencv_hough_lines / pix_opencv_hough_lines

This object detects segments of lines in the incoming frames using the Hough detection algorithm.

Input Parameters :

  mode : detection mode used for the algorithm, one of :
  CV_HOUGH_STANDARD : full lines.
  CV_HOUGH_PROBABILISTIC : line segments.
  CV_HOUGH_MULTI_SCALE : line segments.

  maxlines : maximum number of detected lines.

  threshold : difference threshold for detection of edges.

  minlength : minimal length of a detected segment.

  gap : gap between lines ( for mode CV_HOUGH_PROBABILISTIC )

  aresolution : angle resolution ( for mode CV_HOUGH_MULTI_SCALE )

  dresolution : distance resolution ( for mode CV_HOUGH_MULTI_SCALE )

  nightmode (0/1) : show/hide the incoming video frame.

Output :

  second outlet : for each detected line, coordinates of starting and ending points in the form : 'number xstart ystart xend yend'.

pdp_opencv_hough_circles / pix_opencv_hough_circles

This object detects segments of circles in the incoming frames using the Hough detection algorithm.

Most of the time, the circles do not really exist but are 'almost there'.

Input Parameters :

  maxcircles : maximum number of detected circles.

  threshold :  threshold used in the edges detection ( cvCanny ).

  threshold2 : threshold used for the detection of the centers. 

  mindist : minimum distance between circles centers. 

  resolution : resolution of the accumulator.

  nightmode (0/1) : show/hide the incoming video frame.

Output :

  second outlet : for each detected circles, coordinates of the center and radius in the form : 'number xcenter ycenter radius'.

pdp_opencv_hu_compare / pix_opencv_hu_compare

This object takes two inputs, generally an incoming video stream and a pattern image, also in form of a video stream and tries to detect the pattern in the incoming video frames .

It operates as follows : it extracts from the pattern image the biggest contour and compare the contours of the incoming video using cvMatchShapes, which compares contours calcu lating their Hu moments, characteristics of the topology of the contours.

Input Parameters :

  minsize : minimal size of threated contours, to remove small shapes and noise.

  method : method used in cvMatchShapes which can be one of : 
  CV_CONTOURS_MATCH_I1, CV_CONTOURS_MATCH_I2 or CV_CONTOURS_MATCH_I3 : the difference between the Hu moments are computed with different formulas ( see cvMatchShapes documentat

ion ).

  clear : recalculate the contour of the pattern image if ever it changes.

  criteria : minimal distance under which the contours are considered as detected and highlighted.

Output :

  first outlet : the contours of the incoming video stream with detected contours highlighted.

  second outlet : the pattern image contour.

  third outlet : the minimum distance between the contours of the incoming stream with the pattern image.

  fourth outlet : the coordinates of detected matching contours.

note : as the contours are all normalized before the comparisons, the size and orientation of the contours should not impair the detection, but from the tests we've been running , this is more relevant with the next module that compares contours using a PGH histogram.

pdp_opencv_pgh_compare / pix_opencv_pgh_compare

This object is very similar to the previous one, except that for comparing contours, it uses a PGH histogram instead of the Hu moments.

The PGH histogram is an histogram of the convexities of the contours storing the size and angle of each curve. This method is statistical and is better for detecting contours r egardless of their size and orientation.

Input Parameters :

  minsize : minimal size of threated contours, to remove small shapes and noise.

  clear : recalculate the contour of the pattern image if ever it changes.

  criteria : minimal distance under which the contours are considered as detected and highlighted.

Output :

  first outlet : the contours of the incoming video stream with detected contours highlighted.

  second outlet : the pattern image contour.

  third outlet : the minimum distance between the contours of the incoming stream with the pattern image.

  fourth outlet : the coordinates of detected matching contours.

Template:Start:pdp opencv pgh compare.png

note : this object is less sensitive to the change of size and orientation, but as it is statistical, it can also confuse a contour for another that would include the same numbe r of curves with the same angle.

pdp_opencv_knear / pix_opencv_knear

This object compares the incoming video frames with a collection of samples loaded from a directory.

The idea is to implement a kind of form recognition close to some OCR techniques that recognizes incoming characters calculating the distance from the current input with all the

samples of a database. The calculated distance just calculates the number of pixels that are different after binarizing them.

Input Parameters :

  load <directory> <count>  : load all the samples from a database. The directory must contain files named : 000.png, 001.png....count.png.

  bang : calculates the distance of the incoming video frame with the database.

Output :

  second outlet : the distance of the incoming image with the database.

Here the database is a database of '+' signs and a distance smaller than 2000 detects a more-or-less well-formed sign but this threshold ( 2000 ) depends on the size of the samp les and on the tolerance you want to give to the algorithm, so it must be adapted to each particular case.

pdp_opencv_of_bm / pix_opencv_of_bm

This object, as the next two following ones, calculates the optical flow, trying to correlate the movement of blocks from a frame to next one, this algorithm is known as Block M atching. It uses a variable size of blocks that can be adjusted.

Input Parameters :

 blocksize <width> <height>  : sets the size of the blocks.

 shiftsize <width> <height>  : sets the coordinate increment between blocks ( blocks can be overlapping ).

 maxrange <width> <height>  : sets the size of the pixels scanned around a block ( the actual region of search ).

 threshold <value> : sets the threshold of movement for a single block.

 minblocks <value> : sets the minimum number of blocks that moved over which a global movement is detected.

 useprevious <0/1> : use previous results of movements to calculate the new one ( cumulative movements ).

 nightmode <0/1> : show or hide the original video.

Output :

 second outlet : the average angle of the blocks movement.

 third outlet : the maximum movement of a block with its amplitude and its angle in the form : <amplitude> <angle>.

pdp_opencv_of_hs / pix_opencv_of_hs

This object is very similar to the previous one but does not operate on blocks, rather on each individual pixel. It uses the Horn and Schunck algorithm for the identification of

pixels.

Input Parameters :

 lambda <value>  : sets the value of the Lagrange multiplier.

 threshold <value> : sets the threshold of movement for a single piksel.

 minblocks <value> : sets the minimum number of blocks of one pixel that moved over which a global movement is detected.

 useprevious <0/1> : use previous results of movements to calculate the new one ( cumulative movements ).

 nightmode <0/1> : show or hide the original video.

Output :

 second outlet : the average angle of the blocks movement.

 third outlet : the maximum movement of a block with its amplitude and its angle in the form : <amplitude> <angle>.

pdp_opencv_of_lk / pix_opencv_of_lk

This object is almost the same as the previous one, except that it uses the Lucas/Kanade algorithm for the identification and tracking of points.

Input Parameters :

 winsize <width> <height> : size of the averaging window to group pixels.

 threshold <value> : sets the threshold of movement for a single piksel.

 minblocks <value> : sets the minimum number of blocks of one pixel that moved over which a global movement is detected.

 nightmode <0/1> : show or hide the original video.

Output :

 second outlet : the average angle of the blocks movement.

 third outlet : the maximum movement of a block with its amplitude and its angle in the form : <amplitude> <angle>.

DOWNLOAD & INSTALL

Pdopencv is now version 0.2-rc5 ( october 2009 - codename BITIBITS ).

GNU/Linux

You have the option of installing binary packages or to compile it if you have a special need for it, for example if there are no binary package for your system : gentoo, fedora, ...

Packages for Ubuntu and Debian

 * First install Pd-Extended from the main repository : [[2]].
 
 * For most of the current debian-based systems, pdopencv binary packages are available here : [[3]].
 
 * Just click on the package that corresponds to your system, with the right system version and the kind of CPU you have ( 32 or 64 bits ),
 ( for example for hardy 64 bits use package : pdopencv-0.2.01-20090930-hardy-amd64.deb ), then open the package with the package manager and click 'Install',

next time you start pd, the library should appear in the help browser.

Install from sources

(actually only tested in GNU/Linux Ubuntu Gutsy)

 * first install opencv development packages, on ubuntu :

 apt-get install libcv-dev
 
 apt-get install libcvaux-dev
 
 apt-get install libhighgui-dev
 
 * then get the _SOURCES_ of the pd you are using and of the PDP or GEM that you are loading ( it might not work if the version  is different )

 * download the sources of openCV for PDP [[4]] or for Gem [[http://giss.tv/pd-opencv/pix_opencv-0.2-

rc5.tar.gz|pix_opencv-0.2-rc5.tar.gz]], then compile the externals :

 * unpack it :: 

 tar xzvf pix_opencv-0.2-rc5.tar.gz

 or 

 tar xzvf pdp_opencv-0.2-rc5.tar.gz

 * cd into the library folder ::

 cd pix_opencv

 or 

 cd pdp_opencv

 * configure the package with the appropriate command for your system :

 ./configure --with-pd=<path to pd _sources_> --with-gem=<path to gem _sources_>

 for example :

 ./configure --with-pd=/usr/local/pd --with-gem=/usr/local/pd/gem

 or

 ./configure --with-pd=<path to pd _sources_> --with-pdp=<path to pdp _sources_>
 

 * then, compile it ::

 make clean
 make

 * and finally copy the .pd_linux to your externals folder ::

 cp *.pd_linux /usr/lib/pd/extra//
 cp *.pd /usr/lib/pd/doc/5.reference

next time you start pd, the library should appear in the help browser.

Binary package

 * install Pd-extended from the main repository: http://puredata.info/downloads

 * download: [[5]] from http://www.ient.rwth-aachen.de/cms/software/openc

v/

 * open it and drop the **OpenCV.framework** into your **/Library/Frameworks/** folder

 * download {Template:Start:pix opencv-0.2rc4 macosx-10.5.1 bins.tgz

 * unarchive it and move the *.pd_darwin files to **~/Library/Pd**

next time you start pd, the library should appear in the help browser.

Install from sources

(Although pdp_opencv could work on OSX, we recommend to use the version for GEM for performance reasons : pix_opencv )

first install openCV MacOS framework :

 * download: [[6]] from http://www.ient.rwth-aachen.de/cms/software/openc

v/

 * open it and drop the OpenCV.framework into your /Library/Frameworks/ directory

 * get the _SOURCES_ of the pd you are using and of the GEM that you are loading ( it might not work if the version is different )

 * download the sources of openCV for Gem [[7]],
 then compile the externals :

 * unpack it :: 

 tar xzvf pix_opencv-0.2-rc5.tar.gz

 * cd into the package folder ::

 cd pix_opencv

 * configure the package with the appropriate command for your system :

 ./configure --with-pd=<path to pd _sources_> --with-gem=<path to gem _sources_>

 for example :

 ./configure --with-pd=/usr/local/pd --with-gem=/usr/local/pd/gem

 * then, compile it ::

 make clean
 make

 * and finally copy the .pd_darwin to your externals folder ::

 cp *.pd_darwin /Applications/Pd-*.app/Contents/Resources/extra/Gem/ for example

( of course change the path to the pd version you are using ).

next time you start pd, the library should appear in the help browser.

IMPORTANT TIPS & KNOWN BUGS

In order to make the pix_opencv objects work You always had to set colorspace RGBA. otherwise, You will not see any effect.

FOR DEVELOPPERS

Here you can find some examples of commented code to help you programming with Open CV in pd.

• • pdp_opencv_threshold & pix_opencv_threshold**

• • pdp_opencv_edge & pix_opencv_edge**

(from pdp_opencv_edge_process_rgb) ::

   // We make here a copy of the PDP packet in image->imageData ... 
   // image is a IplImage struct, the default image type in openCV, take a look on the IplImage data structure here
   // http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html 
   memcpy( x->image->imageData, data, x->x_size*3 );
 
   // Convert to grayscale
   cvCvtColor(x->image, x->gray, CV_BGR2GRAY);
 
   cvSmooth( x->gray, x->edge, CV_BLUR, 3, 3, 0, 0 );
   cvNot( x->gray, x->edge );
 
   // Run the edge detector on grayscale
   cvCanny(x->gray, x->edge, (float)x->edge_thresh, (float)x->edge_thresh*3, 3);
 
   cvZero( x->cedge );
 
   // copy edge points
   cvCopy( x->image, x->cedge, x->edge );
 
   //memory copy again, now from x->cedge->imageData to the new data pdp packet
   memcpy( newdata, x->cedge->imageData, x->x_size*3 );_threshold**

(from pix_opencv_edge :: processRGBAImage)

   // Here we make a copy of the pixel data from image to orig->imageData
   // orig is a IplImage struct, the default image type in openCV, take a look on the IplImage data structure here
   // http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html 
   memcpy( orig->imageData, image.data, image.xsize*image.ysize*4 );
 
   // Convert to grayscale
   cvCvtColor(orig, gray, CV_BGRA2GRAY);
 
   cvSmooth( gray, edge, CV_BLUR, 3, 3, 0, 0 );
   cvNot( gray, edge );
 
   // Run the edge detector on grayscale
   cvCanny(gray, edge, (float)this->edge_thresh, (float)this->edge_thresh*3, 3);
 
   cvZero( cedge );
   // copy edge points
   cvCopy( orig, cedge, edge );
 
   //copy back the processed frame to image
   memcpy( image.data, cedge->imageData, image.xsize*image.ysize*4 );

• • pdp_opencv_bgsubstract & pdp_opencv_bgsubstract**

• • pdp_opencv_distrans & pdp_opencv_distrans**

• • pdp_opencv_laplace & pdp_opencv_laplace**

• • pdp_opencv_motempl & pdp_opencv_motempl**

• • pdp_opencv_morphology & pdp_opencv_morphology**

• • pdp_opencv_floodfill & pdp_opencv_floodfill**

• • pdp_opencv_haarcascade & pix_opencv_haarcascade**

• • pdp_opencv_motempl & pix_opencv_motempl**

• • pdp_opencv_contours_boundingrect & pix_opencv_contours_boundingrect**

• • pdp_opencv_lk & pdp_opencv_lk**