Auto Chasing Turtle Hirotaka Niisato Noritsuna Imamura SIProp

Auto Chasing Turtle Hirotaka Niisato Noritsuna Imamura ©SIProp Project, 2006 -2008 1

Agenda Summary Part “Hardware” Need more power supply Nothing display Need more power supply Part “Software” Not Bitmap Calculate the course Calculate the distance ©SIProp Project, 2006 -2008 2

Summary This product is "Auto Chasing Turtle". By autonomous control, this robot recognizes people's face and approaches to the detected human. 1. 2. 3. 4. 5. 6. 7. Rotate and look for the human who becomes a target. Detect the human with recognizing people's face by using RGB camera of Kinect. If it can detect the human, it calculates the course which it should follow. And It direction is changed. Calculate the distance to target by using Z(depth) camera of Kinect. Walk toward to target. 2 -5 are repeated until it becomes a suitable distance. And if losts target, returns to 1. The scene that it is working in real time can be seen by i. Pad. ©SIProp Project, 2006 -2008 3

Movie You. Tube http: //www. youtube. com/watch? v=8 Egf. Ak 5 RBVo Source Code & detail explanation http: //www. siprop. org/ja/2. 0/index. php? product%2 FA uto. Chasing. Turtle ©SIProp Project, 2006 -2008 4

complete set parts Robot Sensors KONDO Animal 01 http: //fswg. oesf. biz/ RGB camera Z(Depth) camera Kinect Controllers For servo RCB 3 For application Beagleboard-x. M Linaro-Kernel Android(Embedded Master) Connect to outside Radio wave Wi-Fi router Viewer i. Pad Power supply 12 V 1 A output 10 V 1 A output 5 V 3 A output ©SIProp Project, 2006 -2008 5

Open. Frameworks x Kinect x Android = ofx. Droid. Kinect ©SIProp Project, 2006 -2008 6

About ofx. Droid. Kinect ©SIProp Project, 2006 -2008 7

About ofx. Droid. Kinect This is the Android Application Framework which runs on open. Frameworks & uses Kinect. Download http: //www. noritsuna. com/archives/2011/01/openframeworks_ kinect_android. html Using softwares open. Frameworks for Android It’s the gaming framework. You can make game easier. http: //openframeworks. cc/ Android Linaro Android http: //git. linaro. org/gitweb? p=people/jstultz/linux. git; a=summary ©SIProp Project, 2006 -2008 8

Part “Hardware” ©SIProp Project, 2006 -2008 9

Need more power supply ©SIProp Project, 2006 -2008 10

Need more power supply Don’t run beagleboard-x. M by 5 V/1 A power supply. It needs 5 V/2 A. Standard USB battery has 5 V/1 A. Create power supply by myself My battery has 12 V/2 A. My plan is to create convertor of 12 V/2 A ⇒ 5 V/2 A. Can’t get power IC… Using car’s cigarette adapter which convert to USB. 12 V・ 24 V/2 A ⇒ 5 V/2 A ©SIProp Project, 2006 -2008 11

Nothing display Can’t view camera’s image Using VNC viewer It needs Wi-Fi router. We attached Wi-Fi router on beagleboard-x. M. ©SIProp Project, 2006 -2008 12

Need more power supply Can’t get IP address from Wi-Fi router needs more “Power supply”. Wi-Fi router: 1 A + beagleboard-x. M: 2 A = Total: 3 A ©SIProp Project, 2006 -2008 13

Part “Software” ©SIProp Project, 2006 -2008 14

Not Bitmap Standard RGB camera(BITMAP) format 32 bit(int) 8 bit(A), 8 bit(R), 8 bit(G), 8 bit(B) 32 bit(int) 8 bit A R G B Kinect’s RGB camera format by libfreenect 3 int 32 bit(R), 32 bit(G), 32 bit(B) 32 bit(int) R 32 bit(int) G B ©SIProp Project, 2006 -2008 15

Not Bitmap However ・・・・・・ can’t detect Face. Must convert format BMP to JPEG… Bitmap bitmap = Bitmap. create. Bitmap(w, h, Bitmap. Config. ARGB_8888); byte[] pixels = OFAndroid. get. Img. Pixels(); if (pixels != null) { for (int i=0; i<w; i++) { for (int j=0; j<h; j++){ int index = (j * w + i) * 3; bitmap. set. Pixel(i, j, Color. rgb(pixels[index++], pixels[index])); } } File. Output. Stream fos = new File. Output. Stream("/screenshot. jpg"); bitmap. compress(Bitmap. Compress. Format. JPEG, 100, fos); fos. flush(); fos. close(); ©SIProp Project, 2006 -2008 16

Calculate the course 1. Calculate the course which it should follow. 2. It direction is changed. Face. Detector detector = new Face. Detector(w, h, faces. length); int num. Faces = detector. find. Faces(bitmap, faces); if (point. X > 0 && point. X < w/4) { Droid. Bot. get. Instance(). turn. Right 2(); // right position } else if (point. X >= w/4 && point. X <= 3*w/4) { ; // center position } else if (point. X > 3*w/4 && point. X <= w) { Droid. Bot. get. Instance(). turn. Left 2(); // left position } ©SIProp Project, 2006 -2008 17

Calculate the course ©SIProp Project, 2006 -2008 18

Calculate the course Using height. Change Kinect’s angle. int angle = 30 - point. Y*30/h; if (angle > 0 && angle <= 30) OFAndroid. set. Angle(angle); ©SIProp Project, 2006 -2008 19

Calculate the distance to target by using Z(depth) camera of Kinect. Range 0~65565 int dist = OFAndroid. get. Distance(point. X, point. Y); if (dist < 100) Droid. Bot. get. Instance(). walk. Back 4(); else if (dist >= 100 && dist < 150) Droid. Bot. get. Instance(). walk. Toward 4(); else if (dist >= 150 && dist < 200) Droid. Bot. get. Instance(). walk. Toward 8(); else if (dist >= 200 && dist < 300) Droid. Bot. get. Instance(). walk. Toward 16(); else if (dist >= 300) Droid. Bot. get. Instance(). walk. Toward 32(); ©SIProp Project, 2006 -2008 20

MAKE: style Hardware beagleboard-x. M KONDO Animal Kinect Software ofx. Droid. Kinect Linaro Kernel Android(Embedded Master) 　 Do It Yourself ⇒Do It With Others! ©SIProp Project, 2006 -2008 21