Computer vision and image processing Accessible resources for

Computer vision and image processing Accessible resources for robotics and CS curricula Designed for Pedagogy: Exploring Robot Vision in CS 1 The Institute for Personal Robots in Education Doug Blank, Bryn Mawr College In association with Tucker Balch, Deepak Kumar, Stewart Tansley, Mark Guzdial, Keith O’Hara, and Jay Summet and many others

Introduction Institute for Personal Robots in Education (IPRE) Fluke + Scribbler, hardware Myro, software Learning Computing with Robots Summer workshops at GATech and Bryn Mawr College this summer! See our booth over the next couple of days.

IPRE Overview Mission: Make education more fun and effective through the use of personal robots Audience: All levels, from middle school to graduate school Joint effort hosted at Georgia Tech with Bryn Mawr College and Microsoft Research 3 year seed funding provided by Microsoft Research Special ingredient and hypothesis: A personal robot for every student programmer

Personal Robot Every student gets their own robot Small enough to carry in backpack Cost about the price of a textbook Wireless, controlled from computer Interactive and easy to program Robust Personalizable More than “just a robot”

Personal Robot turn. Left(1, . 5) speak(“Hello, SIGCSE!”)

IPRE Pilot Hardware Kit Featuring Parallax’s Scribbler 6 Light sensors 7 IR sensors Stall sensor Speaker 5 LEDs 2 motors Bluetooth wireless Camera Gamepad

IPRE’s Philosophy The Personal Robot provides the context The needs of the curriculum drive the design of the robot, software, and text The software is easy to pickup, but scales with experience (“pedagogically scalable”) An accessible, engaging environment for new, diverse students Computer Science ≠ programming Computing a social activity Computing a medium for creativity Focus on performances rather than competitions

Scribbler with IPRE Fluke

A CS 1 Assignment: Exploring a Pyramid

“Civic Computing”

Myro is Python Looks like English Indentation matters Interactive Easy to learn, but powerful

Myro is Python Looks like English Indentation matters Interactive Easy to learn, but powerful … Iron. Python gives access to Microsoft Robotics Studio … and Linux and Macintosh via Mono

Python Functions def function. Name(arg 1, arg 2=value): print “hello world!”, arg 1, arg 2 # this is a comment function. Name(1)

Myro, inspired by Media. Comp

Myro, inspired by Media. Comp picture = take. Picture() count = 0. 0 for pixel in get. Pixels(picture): if distance(get. RGB(pixel), get. RGB(pink)) < 100: total. X += get. X(pixel) count += 1 if total. X/count < get. Width(picture)/2: turn. Left(1, . 2) else: turn. Right(1, . 2)

Panorama for i in range(10): show(take. Picture(“gray”)) turn. Left(1, . 2) ONLINE Evaluation www. cse. buffalo. edu/sigcse 08/evaluations Password: sigcse 08

IPRE “Open” Textbook

Connections to Biology and Psychology

www. roboteducation. org Institute for Personal Robots in Education (IPRE) Fluke + Scribbler, hardware Myro, software Learning Computing with Robots If you are interested in working with us to explore robots and education, please let us know.

Analysis so far: Retention? CS 2 Data Structures Enrollment

Analysis so far: Retention! CS 2 Data Structures Enrollment

Hands-on with Myro 10 IPRE Robot kits Explore image processing, or any aspect of robots and education Install software Explore Python, Myro, and Robot

Using Myro 1. Install software on any computer (Mac, Linux, or Windows) 2. Establish Bluetooth connection (via a Serial port) 3. 4. 5. 6. Start Python Load Myro “Initialize” connection with robot Have fun!

1. Install software M yr o

2. Serial Connection over Bluetooth (“com 4”) Serial Bluetooth Adapter o r y M USB Bluetooth Adapter

3. Start Python (IDLE)

4. Load Myro >>> from myro import *

5. Initialize connection to robot >>> from myro import * >>> init(“com 4”)

6. Have Fun! >>> gamepad()

Gamepad controls Pad -----Left/Right Up/Down Button -----1 2 3 4 5 6 7 8 Action ------turn. Left() and turn. Right() forward() and backward() Action ------take. Picture() beep(. 25, 523) beep(. 25, 587) beep(. 25, 659) speak('Hello. My name is Scribby. ') speak('Ouch! I'm a sensitive robot. ') speak('I'm hungry. ') speak('Good bye, for now. ') and stop()

Manual Control >>> from myro import * Myro, (c) 2008 Institute for Personal Robots in Education [See http: //www. roboteducation. org/ for more information] Version 2. 6. 5, ready! >>> init() Waking robot from sleep. . . Hello, I'm Scribby! Left >>> senses() >>> gamepad() 6 5 7 Forward Right 3 8 2 4 Back 1

Let’s get Personal! Name your robot >>> >>> set. Name(“Fifi”) set. Name(“Robbie”) set. Name(“Toaster”) set. Name(“Bob”)

Challenges 1. Is there Life on Mars? 2. Build a Dancing Robot 3. Build a Robo Cockroach

Challenge 1: Is there Life on Mars? “Mars” is the center of the room There is a picture of an alien in the mountains Manually drive your robot to “Mars” Photograph the specimen Is there life on mars?

It’s alive!

Challenge 2: Build a Dancing Robot www. blogguelph. com

Movement forward(speed, seconds) >>> forward(1, 2. 3) backward(speed, seconds) >>> backward(. 5, 4. 6) turn. Left(speed, seconds) >>> turn. Left(1, . 3) turn. Right(speed, seconds) >>> turn. Right(1, . 6)

Dancing >>> forward(1, 1. 4) >>> turn. Left(1, . 3)

Dancing >>> >>> forward(1, 1. 4) turn. Left(1, . 3)

“Square” Dancing >>> >>> forward(1, 1. 4) turn. Left(1, . 3)

range() >>> range(4) [0, 1, 2, 3] >>> for i in range(4): print “Hi” Hi! Hi!

range() >>> for i in range(4): forward(1, 1. 4) turn. Left(1, . 3)

Making Noise beep(duration, freq 1) >>> beep(. 5, 880) >>> beep(. 5, 440) beep(duration, freq 1, freq 2) >>> beep(1, 440, 880)

Making Noise speak(“any string”) >>> speak(“Hello world!”) >>> speak(“What’s up, Doctor Jay? ”) get. Voices() >>> get. Voices() set. Voice(“Name”) >>> set. Voice(“Fred”)

Making Noise play. Song( make. Song(“string”)) >>> play. Song(make. Song(“a 1; c 1/8”)) >>> play. Song(make. Song(“c 4 1; d 3 1; a 5 1”))

Putting it all together Instead of entering the commands one at a time, we can put them all together into a program From the IDLE menu, select File -> New Window Name it by selecting File -> Save as… from the new window Name it “Dance. py”

Python Looks like English Indentation matters Interactive Easy to learn, but powerful

Dance. py for i in range(10): forward(1, 2) backward(1, 2) for i in range(3): beep(. 25, 200) beep(. 25, 400) beep(. 25, 800) beep(. 25, 1600)

Dance. py for i in range(10): forward(1, 2) backward(1, 2) for i in range(3): beep(. 25, 200) beep(. 25, 400) beep(. 25, 800) beep(. 25, 1600) To run your program, select Run -> Run Module from the new window menu

Myro control wait(seconds) >>> wait(. 5) for seconds in timer(10): # do something for 10 seconds while time. Remaining(20): # do something for 20 seconds

Camera >>> picture = take. Picture() >>> show(picture) Click and drag in show window >>> picture = take. Picture(“blob”) >>> count, x, y = get. Blob() >>> configure. Blob(y 1, y 2, u 1, u 2, v 1, v 2)

Robot Bling >>> set. LEDBack(1) >>> set. LEDBack(0)

Challenge 3: Build a Robo Cochroach If you see light, run

Robot Sensors

Senses get. Light()- “left”, “center”, “right” get. Bright() - “left”, “center”, “right” get. IR() - “left”, “right” get. Obstacle() - “left”, “center”, “right”

Reading the Light Sensors get. Light(“left”) get. Light(“center”) get. Light(“right”) get. Light() [657, 1453, 1025] Light sensors

Python functions def function. Name(arg 1, arg 2=value): print “hello world!”, arg 1, arg 2 # this is a comment function. Name(1)

Thank you for your time! For more information see www. roboteducation. org