If you have any questions please feel free
- Slides: 92
If you have any questions please feel free to interrupt me
The vision system for Marie Curie
Main Tasks in our system Image recognition Machine Learning Control of robot’s behavior Environment
Marie Curie will be communicating with Schroedinger’s Cat robot
Interaction of Marie Curie and Cat 1. 2. 3. 4. 5. Marie Curie does not change its main coordinates, she can only move her hands, head and legs, but she remains attached to the desk. Cat can move freely in the area of the stage. Cat should not bump into Marie or furniture of the lab. Marie should know where the cat is located and look to him Cat should know where Marie is located and talk to her.
Ideal view of the ceiling camera Black curtain Ca t shelf equipment Marie Curie equipment
There will be a Kinect camera looking from the ceiling to the stage
The role of the ceiling camera 1. 2. 3. 4. 5. The camera will be attached to the ceiling or will be in some position very high, as high as we can. We have done something similar but the robots were small. The camera should know x, y coordinates of every robot and its orientation (pose) Marie Curie does not change its main coordinates, so it is easy Cat is fast so we have to track the triplet (x, y, )
There is nothing like that in Disneyland All behaviors of robots in commercial theatres are strictly scripted, robots move on rails, they cannot make an error. In our case we have interaction, improvisation, and robots are subject to noisy behaviors. This task is more similar to robot soccer than to existing robot theatres in the world.
There will be another camera looking to faces of the audience We will call it the human-control camera or a front camera
The role of the front camera 1. The camera is attached to the wall near the glass window of theatre, looking towards humans located in the corridor. 2. This camera will look at the audience 3. There are several goals of having this camera 1. 2. 3. 4. Recognizing (x, y, ) of every person that looks at the performance (perhaps not more than 5). Recognizing the emotion on the faces of these people. Recognizing their gestures with hands and legs, full bodies and faces. Use these data to control the behavior of the robots, songs selected, slides selected, lights and other effects.
There is nothing like that in Disneyland All behaviors of robots in Disneyland are strictly scripted. Rarely humans can change robots’ behaviors.
This is a new task for our team Marek Perkowski has never done anything like this before Perhaps nobody in the world has done something like this. This is good as we are doing something new. Hopefully we have done something similar and have a good experience from the past. We were doing ROBOT SOCCER. We will try now to use our past experience and theory for a new task.
Ideal view of the ceiling camera Black curtain Ca t shelf equipment Marie Curie equipment
This is ideal, in reality the image will be much distorted with noise and lightning and geometry Y axis c Ca t Black curtain Yc Xc shelf equipment Marie Curie equipment X axis
The idea of Robot Soccer
3. Robot Soccer and Similar Tasks • Robot Soccer Competition – – Robo. Cup FIRA Remote controlled systems Autonomous robots • Clustering
3. 1 Robot Soccer “Robo. Cup is an international joint project to promote AI, robotics, and related fields. It is an attempt to foster AI and intelligent robotics research by providing a standard problem where a wide range of technologies can be integrated and examined. Robo. Cup chose to use the soccer game as a central topic of research, aiming at innovations to be applied for socially significant problems and industries. The ultimate goal of the Robo. Cup project is: By 2050, develop a team of fully autonomous humanoid robots that can win against the human world champion team in soccer. ” [Robo. Cup 1998]
As you see, it is difficult to approximate every robot with a rectangle. It will be even more difficult in our case.
Overhead Vision Our goal is to start with Overhead Vision (Ceiling Camera) and check how it will work. We may move to more cameras if necessary.
Local Vision
Design Criteria for robot soccer • Controller Hardware: Enable on-board image processing – – Interface to digital camera Incorporate graphics LCD Incorporate user buttons Wireless communication between robots • Sensors: Allow variety of additional sensors: – Shaft encoders – Infra-red distance measurement sensors – Compass module • Software: Flexibility to accommodate for different robot equipment – Operating system Ro. BIOS – Hardware description table HDT
What is AI? Research in Artificial Intelligence (AI) includes: n n design of intelligent machines formalization of the notions of intelligence and rational behavior understanding mechanisms of intelligence interaction of humans and intelligent machines.
Objectives of AI Engineering : costruct intelligent machines Scientific : understand what is intelligence.
Can a robot do these? Understand? Simulate its environment? Act rationally? Collaborate and compete? Display emotions? A bold claim: A team of Robots will beat the FIFA World Cup champions by 2050!
Robo. Cup - Aim ”pushing the state-of-the-art” ”By mid-21 st century, a team of fully autonomous humanoid robot soccer players shall win the soccer game, comply with the official rule of the FIFA, against the winner of the most recent World Cup. TO BOLDLY GO WHERE MAN HAS GONE BEFORE (cf. Star Trek) Formalised Testbed
Do you really believe that a team of Robots could beat the FIFA World Cup champions by 2050? By all accounts this may sound overly ambitious. In fact, if you compare this goal to other ground breaking achievements it is not ambitious at all. The Wright brothers' first airplane was launched and 50 years later man landed on the moon. Even more recently Deep Blue the computer programmed to play chess, played chess grand master Garry Kasparov and won -- roughly 50 years after the deployment of the first computer. It's a long time. Think what has happened since 1950.
Power of AI Is the following AI? In 1997 a computer, Deep Blue, won a chess match with world champion Kasparov. n n n Accident? IBM paid Kasparov to loose? Brute force with no intelligence? So, what is intelligence?
Simulation Turing test (1950)
Chess versus soccer robot Environment State Change Info. accessibility Sensor Readings Control Chess Static Turn taking Complete Symbolic Central Robo. Cup Dynamic Real time Incomplete Non-symbolic Distributed Difference of domain characteristics between computer chess and soccer robots
Intelligent Agents are situated n n Perception of environment Execution of actions Agents can communicate and collaborate n n they can differ than compete and be more or less egoistic/altruistic The agents have: n n n objectives, communications, intentions.
A New Approach Professor Kim from KAIST The founder of Robot Soccer and FIRA president Two organizations: 1. FIRA (earlier) 2. Robo. Cup (larger)
Four Blocks in two PCBs (Printed Circuit Boards) n n Micro-controller (upper PCB) Communication module (upper PCB) Motor and driving circuits (lower PCB) Power (lower PCB) top view front view side view
Importance of Robot Soccer Communication Cooperation Coordination Learning Competence Real Time Robot Soccer Evolution Computer simulations Wheeled brainless robots Wheeled autonomous robots Legged autonomous robots
Robot Soccer Purpose “The number one goal of [robot soccer] is not winning or losing, but accumulating diverse technology. ” Ø - Mr. Dao (Senior VP of Sony Corporation).
FIRA category Miro. Sot 3 robots on 1 team Size : 7. 5 cm * 7. 5 cm Ball : orange golf ball Playground : black wooden rectangular playground n (150 cm * 130 cm * 5 cm) Vision : global vision system n (more than 2 m above playground)
Experimental Setup of the Vision System Control panel
FIRA category Naro. Sot 5 robots on 1 team Size : 4 cm * 5. 5 cm Ball : orange table-tennis ball Playground , Vision : same as Mirosot
FIRA category Khepera. Sot 3 robots on 1 team Ball : yellow tennis ball Playground : green playground (105 cm * 68 cm * 20 cm) Robot : Khepera Robot Vision : K 213 Vision Turret
FIRA category Robo. Sot 3 robots on 1 team Size : 15 cm * 30 cm Ball : red roller-hockey ball Playground : black wooden rectangular playground (220 cm * 150 cm * 30 cm) Vision : on the robot Under preparation
Small. Size League
Small-Size League (F-180) Field: 2. 7 m x 1. 5 m Size Area : 18 cm rule (fit inside in 18 cm diameter cylinder) Height : 15 cm (global vision), 22. 5 cm (otherwise) teams of autonomous small size robot play soccer game on a field equivalent to a ping-pong table. Each team consists of 5 robots.
Small size league The field is the size and color of a Ping Pong table
orange golf ball Robots move at speeds as high as 2 meters/s econd n Global vision is allowed
Robot Soccer Initiative Host comput er Communicati on System Vision system Host computer Communicati on System “Brainless” System Robots on the playing field Basic Architecture for Robot Soccer Systems
Vision System Vision : global vision system (more than 3 m above ground) Each team has its own camera and PC
Small-Size League 20 minutes, 2 breaks
Real Robot Small-Size League Competition
Middle. Size League
Middle-size Real Robot League (F-2000): Local VISION n The field is the size and color of a 3 x 3 arrangement of Ping Pong tables (9 -3 5 -meter field) n Each team consists of 5 robots playing with a Futsal-4 ball (4 players, one goal-keeper) n Larger (50 centimeters in diameter) robots n Global vision is not allowed. w Each robot has its own vision system n Goals are colored n Field is surrounded by walls to allow for distributed localization through robot sensing n Rule structure based on the official FIFA rules
Medium size league Teams of autonomous mid size robots
Real Robot Middle-Size League Competition Ball : red small soccer ball (FIFA standard size 4 or 5) Playground : green playground (10 m * 7 m * 0. 5 m)
Medium Size League
Medium Size League
Robots can be heterogenous
Middle-Size League
Sony Legged Robot League
Sony Legged Robot League 3 robots on 1 team (including the goalkeeper). Robot : AIBO ERS-110 (provided by Sony)
No communication, autonomous robots, software only. Legged Robot League. 2. 8 m x 1. 8 m 2 players and 1 goal-keeper in a team
Sony Legged Robot League Is played on a field, approx 3 x 2 meter Sony develops the robots, and provides a interface for the programming of the robots.
No Hardware modification is allowed Playing time is 10 minutes per half, with a 10 minute break at halftime
Do different Robots have different personalities? Some teams have robots with very different capabilities. But it is hard to think of them as having personalities; n rather the robots have different playing styles.
Early Sony prototype
n Robot movements closely mirror those of animals
The winner is the team that scores the most goals. In the event of a tie, a sudden death penalty kick competition will determine the winner
The Legged Robot League
The Legged Robot League If opposing teams' robots are damaged or play is excessively rough (whether intentional or not), penalties may be assessed to the offending robot
Humanoid League
Starting 2002, the humanoid league
Humanoid League Bi-Ped League (Humanoid) n n Australia Japan
Where is the science in these robot competitions? Global vision Local vision Other sensors Cooperation Sensor fusion Strategy Learning
Sensors and Actuators for Robot Soccer Local and Global VISION
Sensors for Robot Soccer • Shaft Encoders – PI controller to maintain wheel speed – PI controller to maintain path curvature – Dead reckoning for vehicle position + orientation • Infrared Distance Measurement – Avoid Collision – Navigate and map unknown environment – Update internal position in known environment • Compass – Update orientation independent of shaft encoders – Fault-tolerance in case robot gets pushed or wheels slip
Sensors for Robot Soccer • Digital Camera – Low resolution, 60 x 80 pixels, 24 bit color (Braunl) – Color or shape recognition • Communication – Sharing information among robots – Receiving commands from human operator
VISION: Color Detection • In robot soccer, objects are color coded: n n n ball, goals, opponents, team mates, walls, etc. Teach ball and goal color (hue) before starting the game Match colors in HSI space → Better in changing lighting conditions
Role of Vision Brain-on-board system Robots The robots have functions such as velocity control, position control, obstacle avoidance, etc. Host computer The host computer processes vision data and calculates next behaviors of robots according to strategies and sends commands to the robots using RF modem.
2. 2 Robot-based system Distributed system Intelligent part is implemented in the robots. Suitable when the large number of agents exist Complex and expensive Need communication among robots
Role of vision Robot-based system Robots The robots decide their own behavior autonomously using the received vision data, own sensor data and strategies. Host computer The host computer processes only vision data can be considered as a kind of sensor.
System Comparisons Merits Remote-brainless system n Robot -based system n Brain-on-board system n n n Low cost Easy to develop Suitable for many agents Can use local information Suitable to modularize Demerits n n n Cannot use local sensors High computing power & fast sampling time Complex and expensive robots. Hard to build the system Risk of inconsistent property between host computer and robot system Research purpose n n n Vision system Multi-agent theory Robot system Multi-agent system development Robot-based and vision-based systems
VXD: role of color Initialization n n Click ‘Load VXD’ in the Initialize group box Click ‘Start Grab’ Configuration n n n ‘Load Conf. ’: load a configuration file ‘Save Conf. ’: save current configuration ‘Set Robot Size’: set the robot size in number of pixels ‘Set Pixel Size’: set the size of each color (ball, team, robot, opponent) patch in number of pixels ‘Set Boundary’: set the field boundary on the screen ‘Change Color’: change the color setting of each color patch ‘Set Color’: set the range of tolerance of each color
Subsystems and Vision Serial Port n Select the serial communication port Home Goal n Select the home side on the screen Find Objects n Check the box of which you like to find on the field Initial Position: tell the vision system the initial position of each object n n E. g. ) for the ball i) turn on the radio button of ‘Ball’ ii) place the mouse on the ball and press the left button Repeat above procedure for another object
Commands for Vision Select Situation n The situation in which the game is about to start Command n n n Click ‘Ready’: the vision system starts finding the objects on the field Click ‘Start’ : the vision system starts sending commands to the robots Click ‘Stop’ : the vision system stops finding objects and sending commands
Tasks for us How to organize the ceiling camera system? How to describe (learn? ) the shapes of robots? How to find the (x, y, ) for each robot? How to modify the scripted behavior when the triplets for each robot are known? How to design the interactive behaviors?
Task for Robot Theatre Team For next week Write a half-page essay about the vision system that we discussed today. Use you knowledge from other lecture of today. Add your imagination and crazy ideas about what the robots should see and know for our particular scene of Marie Curie and Schroedinger’s Cat.
- Please feel free to interrupt me
- Please contact me if you have any questions
- Thank you for your attention do you have any questions
- Please feel free to modify
- Please feel free to use it
- Feel free to call
- Will you please be quiet please summary
- Pertanyaan terbuka
- What do you do in your free time جواب
- To feel sad to feel thrown down in spirit
- Because you have rejected me i have rejected you
- Have any questions
- How do you feel when you
- Possessives quiz
- Do you have any apples
- As-tu un animal in english
- Done with work today
- Free hearts free foreheads you and i are
- Rail transport disadvantage
- Thank you for you listening
- Thank you for listening any questions
- Who sang i feel free
- Training feel free
- There are some cake
- Any to any connectivity
- I have 6 faces 8 vertices and 12 edges
- Good morning please have a seat
- Good evening have a seat
- Ladies and gentlemen can i have your attention please song
- Good afternoon please have a seat
- Good morning please have a seat
- Sit in your seat
- Please come in and sit
- Good morning please have a seat
- Where have you gone charming billy questions
- Have you ever wondered questions and answers
- Have you ever thought about questions
- Have you ever questions
- Places you can feel your pulse
- Microscope madness answer key
- Today i feel because
- How we feeling today
- Figurative language in the song happy
- Did you feel it hawaii
- Can you feel it in the air in the air
- Google imagens
- Feel like a blob
- Do you ever feel already buried deep
- Why do you feel proud of mt everest and gautam buddha
- Implied texture
- Self esteem definition
- Can you feel air
- Period or pregnant
- Please type your questions in the chat box
- Activity 3 answer the following questions
- Aphrodite strengths and weaknesses
- An inspector calls act 1 quiz
- If you had studied hard you would have succeeded
- Direct quotations
- You must unlearn what you have learned
- The triangle shirtwaist factory fire commonlit answer key
- You have more potential than you think
- May you be happy in the life you have chosen
- Where are you going where have you been true story
- Conditional perfect continuous tense
- Hawk roosting annotations
- If i had 3 wishes they would be
- Are you sure you have a strategy
- Gibbs free energy equation
- Negative free energy change
- Gibbs free energy non standard conditions
- Allocating kernel memory in os
- Helmholtz free energy and gibbs free energy
- A story of an hour summary
- Free free absorption
- Could you please tell me where
- Please clean your room before we leave for school
- The church of please and thank you
- Excuse me would you please tell me
- Paint must never hope to reproduce the faint
- Could you please tell me where is my uncle's room
- Please clean the room before you live
- Please clean the room before you live
- Yes clean your room
- Can you please turn off
- Help me you to the put picture can here please
- Please read the carefully before you fill out the form
- Would you please tell me where the mall is
- Could you tell me please
- Please sit at the table
- Did you finish your homework?
- Why critical reading is an active process of discovery
- Any queries