EEC492592 Kinect Application Development Lecture 17 Wenbing Zhao

EEC-492/592 Kinect Application Development Lecture 17 Wenbing Zhao wenbing@ieee. org

Outline n Emgu CV q q q Software installation Configure a new project with Emgu CV Working with Emgu CV images

Software Needed n n Open. CV does not support C#: cannot use it directly We will be using Emgu CV: a cross platform. Net wrapper for Open. CV q http: //www. emgu. com/wiki/index. php/Main_Page q Due to unknown reason, later version of Emgu. CV does not work on x 64 Windows platform q Latest version that worked for me is: http: //sourceforge. net/projects/emgucv/files/emgucv/2. 2. 1/libemgucv-windows-x 64 -2. 2. 1. 1150. zip/download

Configuration with Emgu. CV n n n It is tricky to use Emgu. CV in Visual Studio 2010 You must add the following Emgu. CV dll files as references q Emgu. CV. dll q Emgu. CV. UI. dll q Emgu. Util. dll Furthermore, you must copy at least the following Open. CV dlls to your project’s bin/Debug or bin/Release directory q opencv_core 220. dll q opencv_imgproc 220. dll

Configuration with Emgu. CV n n Alternatively, you can add opencv dlls as files into your project. For each dll, right click and select properties. Then choose “copy always”. The dll will be copied to the output directory for you. More one critical step. Right click your project name in the “Solution Explorer” pane, and select properties. Then, choose the “Build” tab and change the platform target to “x 64”

Configuration with Emgu. CV

Working with Emgu. CV Images n n Emgu. CV Image Class Creating Image Getting/setting pixels Drawing objects on image 1/15/2022 EEC 492/693/793 - i. Phone Application Development 7

Generic Parameters for Emgu. CV Supported TDepth types: Image<TColor, TDepth> Images Byte Depth: number of bits used to represent n q q the color of a single pixel Color: type of color representation. Emgu CV supports Supported TColor types: Gray Bgr (Blue Green Red) Bgra (Blue Green Red Alpha) Hsv (Hue Saturation Value) Hls (Hue Lightness Saturation) Lab (CIE L*a*b*) Luv (CIE L*u*v*) Xyz (CIE XYZ. Rec 709 with D 65 white point) Ycc (YCr. Cb JPEG) SByte Single (float) Double UInt 16 Int 32 (int) Image<Gray, Byte> image = new Image<Gray, Byte>( width, height);

Creating Emgu. CV Images n Creating a new image q No background Image<Bgr, Byte> img 1 = new Image<Bgr, Byte>(480, 320); q With blue background Image<Bgr, Byte> img 1 = new Image<Bgr, Byte>(480, 320, new Bgr(255, 0, 0)); n Creating an image from an existing file Image<Bgr, Byte> img 1 = new Image<Bgr, Byte>("My. Image. jpg"); n Creating an image from bitmap Image<Bgr, Byte> img = new Image<Bgr, Byte>(bmp); 1/15/2022 EEC 492/693/793 - i. Phone Application Development 9

Accessing Image Data (Pixels) n Direct access to Image //Color Image My_Image[0, 0] = new Bgr(Color. Red); //Gray Image gray_image[0, 0] = new Gray(200); //Color Image Bgr my_Bgr = My_Image[0, 0]; //Gray Image Gray my_Gray = gray_image[0, 0];

Accessing Image Data (Pixels) n Access via Image. Data property q q Must respect the image depth when accessing the Data property A grayscale image has a depth of one (i. e. , 1 bit) n q [x, y, 0] RGB color image has a depth of 3 (i. e. , 23 bits or one Byte) n [x, y, 0], [x, y, 1], [x, y, 2] represent the blue, green, and red spectrums //Color Image Color R = Color. Red; My_Image. Data[0, 0, 2] = R. R; //Write to the Red Spectrum My_Image. Data[0, 0, 1] = R. G; //Write to the Green Spectrum My_Image. Data[0, 0, 0] = R. B; //Write to the Blue Spectrum //Gray Image gray_image[0, 0] = new Gray(200);

Accessing Image Data: Examples Using column per row access: http: //stackoverflow. com/questions/5101986/iterate-over-pixels-of-animage-with-emgu-cv n private void setting. Pixels. Slow(Image<Bgr, Byte> img) { Stopwatch evaluator = new Stopwatch(); int repetitions = 100; Bgr color = new Bgr(100, 40, 243); evaluator. Start(); for (int run = 0; run < repetitions; run++) { for (int j = 0; j < img. Cols; j++) { for (int i = 0; i < img. Rows; i++){ img[i, j] = color; } } } evaluator. Stop(); Console. Write. Line("Average execution time for {0} iteration n using column per row access: {1}msn", repetitions, evaluator. Elapsed. Milliseconds / repetitions); }

Accessing Image Data: Examples n Using Data property private void setting. Pixels. Faster(Image<Bgr, Byte> img) { Stopwatch evaluator = new Stopwatch(); int repetitions = 100; evaluator. Start(); for (int run = 0; run < repetitions; run++) { for (int j = 0; j < img. Cols; j++) { for (int i = 0; i < img. Rows; i++) { img. Data[i, j, 0] = 100; img. Data[i, j, 1] = 40; img. Data[i, j, 2] = 243; } } } evaluator. Stop(); Console. Write. Line("Average execution time for {0} iteration n using Data property: {1}msn", repetitions, evaluator. Elapsed. Milliseconds / repetitions); }

Accessing Image Data: Examples n Using Data property outside the loop: private void setting. Pixels. Fastest(Image<Bgr, Byte> img) { Stopwatch evaluator = new Stopwatch(); int repetitions = 100; byte[, , ] data = img. Data; evaluator. Start(); for (int run = 0; run < repetitions; run++) { for (int j = 0; j < img. Cols; j++) { for (int i = 0; i < img. Rows; i++) { data[i, j, 0] = 100; data[i, j, 1] = 40; data[i, j, 2] = 243; } } } evaluator. Stop(); Console. Write. Line("Average execution time for {0} iteration n using Data property smartly: {1}msn", repetitions, evaluator. Elapsed. Milliseconds / repetitions); }

Automatic Garbage Collection n n The Image<TColor, TDepth> class automatically take care of the memory management and garbage collection Once the garbage collector decided that there is no more reference to the Image<TColor, TDepth> object, it will call the Disposed method, which release the unmanaged Ipl. Image structure using (Image<Gray, Single> image = new Image<Gray, Single>(1000, 800)) {. . . //do something here in the image } //The image will be disposed here and memory freed

First Emgu. CV WPF App n Basic image type is: Image<Rgb, byte> q n Image<> offers many constructors q q q n It is a wrapper for Ipl. Image class in Open. CV public Image(Bitmap bmp); public Image(string file. Name); public Image(int width, int height, TColor value); public Image(int width, int height, int stride, Int. Ptr scan 0); In the app, we will create a new Image object, draw “Hello world”, and display via WPF image control

First Emgu. CV WPF App n n n n Create a new WPF project Add emgu cv dlls and opencv dlls Add System. Drawing reference Change platform target to x 64 Add one image control in Main. Window Draw “Hello world” using Emgu. CV Image<> Convert the image to Bitmap. Source using the provided helper function Set the Bitmap. Source as the WPF image control source

First Emgu. CV WPF App n Add the following namespaces: q q q using Emgu. CV; using Emgu. CV. Cv. Enum; using Emgu. CV. Structure; // for Image to Bitmap. Source conversion q using System. IO; q using System. Runtime. Interop. Services; q using System. Drawing. Imaging; // for Stopwatch q using System. Diagnostics;

First Emgu. CV WPF App public Main. Window() { Initialize. Component(); //Create an image of 400 x 200 of Blue color Image<Bgr, Byte> img = new Image<Bgr, byte>(400, 200, new Bgr(255, 0, 0)); MCv. Font f = new MCv. Font(FONT. CV_FONT_HERSHEY_COMPLEX, 1. 0); //Create the font //Draw "Hello, world. " on the image using the specific font img. Draw("Hello, world", ref f, new System. Drawing. Point(10, 80), new Bgr(0, 255, 0)); Bitmap. Source imgsource=To. Bitmap. Source(img); image 1. Source = imgsource; }

First Emgu. CV WPF App // taken from: http: //blogs. claritycon. com/blog/2012/11/blob-tracking-kinectopencv-wpf/ [Dll. Import("gdi 32")] private static extern int Delete. Object(Int. Ptr o); public Bitmap. Source To. Bitmap. Source(IImage image) { using (System. Drawing. Bitmap source = image. Bitmap) { Int. Ptr ptr = source. Get. Hbitmap(); //obtain the Hbitmap Bitmap. Source bs = System. Windows. Interop. Imaging. Create. Bitmap. Source. From. HBitmap( ptr, Int. Ptr. Zero, Int 32 Rect. Empty, System. Windows. Media. Imaging. Bitmap. Size. Options. From. Empty. Options()); Delete. Object(ptr); //release the HBitmap return bs; }}

Have a little more fun n n n n Emgu. CV Image also provides a number of drawing methods Draw a Circle of the specific color and thickness // thickness: If thickness is less than 1, the circle is filled up void Draw(Circle. F circle, TColor color, int thickness); // others void Draw(Ellipse ellipse, TColor color, int thickness); void Draw(IConvex. Polygon. F polygon, TColor color, int thickness); void Draw(Line. Segment 2 D line, TColor color, int thickness); void Draw(Rectangle rect, TColor color, int thickness);

Accessing Pixels in Image n n Try out the three ways of accessing pixels in the image Draw more customized patterns in the image