Mainska vizija Dr Nenad Jovii 2019 tnt etf

Mašinska vizija Dr Nenad Jovičić 2019. tnt. etf. rs/~mv

Kamera na sortirnoj liniji BASLER A 641 fc

Kontrola akvizicije • Generalno kod industrijskih kamera postoje dva režima akvizicije: – Snap – jedna slika i jedan streaming – Grab – niz slika i konstantan streaming • Pojedinačna slika ili niz slika u sekvenci se uzorkuje generisanjem trigger signala: – Softverski – komanda sa računara – Hardverski – spoljašnji trigger signal ili interni periodični trigger.

Ex. Trig signal Hardverski trigger

Trigger->Exposure->Readout

Model industrijske linije za sortiranje primenom mašinske vizije • Projekat MV_2019_Sorting. Line_Template Pylon + Open. CV + serial machine interface

Arhitektura sistema • Osnovni principi: – Hardver tj. kontroler sortirne linije upravlja linijom i trigeruje kameru (opciono). – Kontroler vodi računa o detekciji objekta, tačnim pozicijama početka i kraja objekta, i o modulaciji brzine trake. – PC kroz serijsku komunikaciju od kontrolera dobija informaciju o detekciji objekta, i zadaje poziciju zaustavljanja.

Arhitektura softvera • Pylon – podešavanje parametara kamere i upravljanje akvizicijom slike • Open. CV – imlementacija osnovnih algoritama za obradu slike (konverzija kolor sistema) • Serijska komunikacija – komunikacija sa hardverom tj. kontrolerom sortirne linije

Osnovna mašina stanja

Osvetljenje

Pristupi u projektovanju osvetljenja • Pomeraj i gledaj (najčešći metod) – Testiranje sa priručnim osvetljenjem pod različitim uglovima dok se slika prati uživo. . . • Naučni pristup (najefektivniji) – iskustvo + znanje + čeklista upućuju na rešenje • Korišćenje testnih komponenti (najbrže ali i najskuplje) – Isprobati različite profesionalne komponente za osvetljenje u laboratoriji

Sve je u kontrastu • Uspešno projektovano osvetljenje: – Ističe objekte od interesa, tj. povećava kontrast tih objekata u odnosu na okolinu, ili povećava kontrast unutar samih objekata – Potiskuje objekte koji nisu od interesa, tj. umanjuje njihov kontrast (pozadina, šum. . . ) – Obezbeđuje minimalnu osetljivost na varijacije usled: • Minimalnih razlika od uzorka do uzorka • Pristustvo ambijentalnog osvetljenja • Pozicije uzorka u ondosu na sistem za snimanje

Sve je u kontrastu • Uspešno projektovano osvetljenje: – Ističe objekte od interesa, tj. povećava kontrast tih objekata u odnosu na okolinu, ili povećava kontrast unutar samih objekata – Potiskuje obejkte koji nisu od interesa, tj. umanjuje njihov kontrast (pozadina, šum. . . ) – Obezbeđuje minimalnu osetljivost na varijacije usled: • Minimalnih razlika od uzorka do uzorka • Pristustvo ambijentalnog osvetljenja • Pozicije uzorka u ondosu na sistem za snimanje

Šta je potrebno znati • Znanje o: – Prednosti i mane različitih tehnika osvetljenja – Spektralne karakteristike senzora (kamere) – Primenljivost različitih tehnika osvetljavanja u zavisnosti od karakteristika materijala – Zahtevi i ograničenja različitih tehnika osvetljavanja • Razumevanje četiri kamena temeljca osvetljenja u viziji: – – Geometrija Struktura svetlosti Spektar tj. boja svetlosti filtriranje • Detaljna analiza: – Fizički uslovi u kojima je potrebno primeniti osvetljenje – Način na koji intereaguju svetlost i uzorak u konkretnoj aplikaciji

Svetlosni izvori • Širokopojasni – – LED - Light Emitting Diode Quartz Halogen Fluorescent Xenon (Strobing) _______________ • Diskretni (Microscopy) – Metal Halide – High Pressure Sodium – Mercury

Izvori – spektar Quartz Halogen / Tungsten Mercury (Purple) 100 Daytime Sunlight Relative Intensity (%) 80 Fluorescent 60 White LED Xenon 40 20 0 Red LED 300 400 500 Wavelength (nm) 600 700

LED

LED - snaga i trajnost

Koja je dioda sjajnija?

Metal-halide lampe

Živine lampe

High pressure sodium lampe

Poređenje standardnih izvora Type LED Fluorescent Halogen Xenon Spectrum Intensity Life (hrs) Comments Various Bright to Very Bright Up to 100, 000 Long life Stable Output Small Area ** White w/bluegreen, yellow Bright 5000 to 7000 Inexpensive Need High Freq Low Heat White w/yellow Very Bright 200 to 3000 Inexpensive High Heat White w/blue Very Bright 3000 to 7000 Expensive Stable

Uporedne karakteristike

Osetljivost senzora na spektar svetlosti IR Enhanced Analog 80 Digital Interline Transfer Standard Analog CMOS Absolute QE (%) 60 UV Enhanced Analog Human Scotopic (Night-time) Human Photopic IR Block (Short Pass) 40 20 0 300 400 500 600 700 Wavelength (nm) 800 900 1000

Osnovne tehnike osvetljavanja Partial Bright Field Diffuse Dome Dark Field Axial Diffuse Full Bright Field Back Lighting Flat Diffuse

Bright field vs Dark field Typical Co-axial Ring Light – Sample Geometry Bright Field Dark Field

Dark field vs Bright field Partial Bright Field Lights in White Area Dark Field Lights in Grey Areas Mirrored Surface Scratch

Dark field vs Bright field Bright Field Lighting Dark Field

Difuzno DOME osvetljenje • Similar to the light on an overcast day. • Creates minimal glare.

Aksialno difuzno osvetljenje • Light directed at beam splitter • Used on reflective objects • Surface Texture Is Emphasized • Angled Elevation Changes Are Darkened

Ravno difuzno (flat diffuse) osvetljenje • Diffuse sheet directed downward • Long WD and larger FOV • Hybrid diffuse (dome and coaxial) Diffuse Coaxial BF Ring Coaxial Diffuse Flat Diffuse Coaxial DFDome Ring

Primer – tekst na limenci • Concave, reflective surface Axial Bright Diffuse Field Illuminator Ring. Dome Light Dark Field Ring Light Diffuse

Primer – tekst na metalu • Recessed metal part • Reflective, textured, flat or curved surface Bright field. Line spotlight Bright field ring Dark Field ring light

Primer – nalepnica sa celofanom Reading under Cellophane Axial Illuminator Broad Bright Dark. Diffuse Area Field. Linear Ring. Light Array

Tehnika u zavisnosti oblika i teksture Surface Reflectivity Matte Mixed Flat Geometry Independent Area Axial Diffuse Bright Field Flat Diffuse Surface Texture / Shape Uneven Topography Dark Field Diffuse Dome / Cylinder Curved Mirror Specular

Pozadinsko osvetljenje • Locates edges – Gauging • Internal defects in translucent parts • Hole-finding • Presence / Absence • Vision-Guided Robotics: Incl. Pick & Place • Useful for translucent materials

Pozadinsko osvetljenje Light Diffraction: Bending around obstacles Q = l / D, where Q is the diffraction angle and D is opening width High-accuracy gauging: Use monochrome light Shorter wavelengths best Q Q Use collimation – parallel rays D Red Longer l light penetrates D Blue samples better

Pozadinsko osvetljenje - primer • Small Bottle – blue-green • Consider colors and materials properties also. • Longer wavelength isn’t always best for penetration! 880 nm IR Backlight 660 nm Red Backlight 470 nm Blue Backlight

Usmereno pozadinsko osvetljenje No Collimation Film Collimation

Pozadinsko osvetljenje u službi preciznih merenja • Back lighting • Monochrome light better – Shorter wavelength a little better • Collimation even better – Optical collimation better than film • Minimal distortion lens – Telecentric lens best • Measurement calibration – CRITICAL* • Focus - CRITICAL*

Korišćenje boje Use Colored Light to Create Contrast Warm • Use Like Colors or Families to Lighten Cool R (green light makes green features V brighter) • Use Opposite Colors or Families to Darken darker) O B (red light makes green features Y G

Povećavanje kontrasta Warm Red Cool R Green V O B Y Blue White Consider how color affects both your object and its background! White light will contrast all colors, but may be a contrast compromise. G

Povećavanje kontrasta • Candy pieces are imaged under (a) white light and a color CCD camera, (b) white light and a black and white camera, (c) red light, lightening both the red and yellow and darkening the blue, (d) red and green light, yielding yellow, lightening the yellow more than the red, (e) green light, lightening the green and blue and darkening the red, (f) blue light, lightening the blue and darkening the others.

Imaging Beyond “Visible” – Near IR • Infra-red (IR) light interacts with sample material properties, often negating color differences. White light – B&W Camera IR light – B&W Camera

Imaging Beyond “Visible” – Near IR • Near IR light can penetrate materials more easily because of the longer wavelength. Red 660 nm Back Light IR 880 nm Back Light

Imaging Beyond “Visible” – Near UV • Near UV light when used w/ a matched UV excitation dye, illuminates codes and structural fibers. ØTop Image Set: Diaper ØLower Image Set: Motor Oil Bottle

Ambient Light: Any light, other than the visionspecific lighting that the camera collects. - Overhead plant lighting Mercury, HP Sodium, Fluorescent Tubes - Other nearby task lighting Incandescent, Fluorescent Tubes - Indicator status lights - Temporary lighting – construction, emergency - Sunlight – Weather and time-dependent - Interference from other nearby vision-specific lighting!

Ambient Light Controlling and Negating Ambient Light Turn off the ambient contribution Most effective. . . Least Likely! Overwhelm the ambient contribution w/ strobing Effective, but requires more cost and complexity Build a shroud Very effective, but time-consuming, bulky and expensive Control it with pass filters Very effective, but requires a narrow-band source light

Pass Filters in Machine Vision • Pass filters exclude light based on wavelength. Reduce sunlight and mercury vapor light 4 X 510 nm Short Pass 715 nm Long Pass Reduce fluorescent light 35 X 660 nm Band Pass

Pass Filters • Top Image: UV light w/ strong Red 660 nm “ambient” light. ØBottom Image: Same UV and Red 660 nm “ambient” light, with 510 nm Short Pass filter applied.

Avoiding Surface Glare ØChange Geometry – 3 D spatial arrangement of Light, Sample, and Camera (preferred) ØStrobe to overwhelm glare from ambient sources ØUse polarization filters (least preferred) Courtesy Wikimedia Commons

Avoiding Surface Glare 3 -D Reflection Geometry: Light - Sample - Camera

Polarizing Filters in Machine Vision Coaxial Ring Light w/o Polarizers Coaxial Ring Light w/ Polarizers Off-Axis Ring Light w/o Polarizers Coaxial Ring Light w/ Polarizers

Polarizing Filters in Machine Vision Back Light - No Polarizer Back Light - Crossed Polarizers

Polarizing Filters in Machine Vision 6 -pack Plastic Ring Carrier Back Light w/o Polarizers Polarized backlighting is best used to detect internal anisotropy in transparent materials. Back Light w/ Polarizers

• • • • • 1. Immediate Inspection Physical Environment Physical Constraints Access for camera, lens, and lighting in a 3 D space (working volume) The size and shape of the working volume Min and max camera, lighting working distance and fieldofview Part Characteristics Sample stationary, moving, or indexed? If moving or indexed, speeds, feeds, and expected cycle time? Strobing? Expected pulse rate, ontime, and duty cycle? Are there any continuous or shock vibrations? Is the part presented consistently in orientation and position? Any potential for ambient light contamination? Ergonomics and Safety Manintheloop for operator interaction? Safety related to strobing or intense lighting applications?

Standard Lighting Method Ø Determine the Exact Features of Interest Ø Analyze Part Access / Presentation § Clear or obstructed, Moving / Stationary § Min / Max WD range, Sweet Spot FOV, etc. Ø Analyze Surface Characteristics § § Texture Reflectivity / Specularity Effective Contrast – Object vs. background Surface flat, curved, combination Ø Light Types and Applications Techniques Awareness § Rings, Domes, Bars, ADIs, Spots, Controllers § Bright Field, Diffuse, Dark Field, Back Lighting Ø Determine Cornerstone Issues § 3 -D Geometry, Structure, Color & Filters Ø Ambient Light Effects / Environmental Issues

Summary and Conclusions Develop the lighting solution early on in the vision system process Determine appropriate light geometry techniques Consider reflection geometry Be aware of and block ambient light Consider camera wavelength sensitivity Use monochrome light for high-accuracy gauging Remember that light MAY interact differently w/ respect to surface texture, color and composition Make the lighting solution robust Need more help? – Call your lighting professional!!