Introducing Computer Programming with Multimedia Barb Ericson Institute

Introducing Computer Programming with Multimedia Barb Ericson Institute for Computing Education Georgia Institute of Technology ericson@cc. gatech. edu

Computer Science Crisis • 40% drop in majors since 2001 Nationwide – Up to 60% drop in some places • Percentage of women is down to about 12% – From a high of 30% in the 1980 s • Never attracted many African Americans, Hispanics, Native Americans

Why the Decline? • Dot com crash in 2001 – Loss of jobs • AP Exam switched from C++ to Java – Many schools quit teaching AP Computer Science • Visual Basic changed – In. net to be object-oriented • Perceptions of Computer Science – boring, antisocial, not creative, and just programming

Looming Shortage • Only a few jobs have gone overseas – About 2 -3% • We have recovered from the dot com crash – There are now more jobs in this area than in 2001 • Computing jobs are some of the fastest growing – US Dept of Labor predicts 45. 93% growth

Reversing the Trend • We need to get more students interested in computer science – Not drive them away • We need to appeal to a diverse group of students – Not just geeks • All students need to have some understanding of computer science – Our society relies on technology

Our Solution • Use multimedia projects to introduce computing concepts – Pictures, sounds, movies, text • Use open-ended projects to encourage creativity – Create an image collage, create a sound collage, create a movie • Make computing social – Encourage students to show off their work

Choice of Languages • Python – Used by many companies: IL&M, Google, Nextel, etc. – Easier syntax than many languages – Can use to teach functional, procedural, or objectoriented programming – Used as a scripting or extension language for many applications – Growing in popularity • Java – Popular language used for AP CS

Digitizing Pictures • We digitize pictures into lots of little dots • Enough dots and it looks like a continuous whole to our eye • Each picture element is referred to as a pixel – Each pixel knows its color – It also knows where it is in its picture

Encoding Color: RGB • In RGB, each color has three component colors: – Red, Green, and Blue • In most computer-based models of RGB, a single byte (8 bits) is used for each – So a complete RGB color is 24 bits, 8 bits of each – 0, 0, 0 is black – 255, 255 is white – 255, 0, 0 is red

Make and Show a Picture Use pick. AFile function To select a picture file (ends in. jpg) Use make. Picture(file) to create the picture Use show(picture) to display the picture

Use Media. Tools • Click on Media. Tools • Select the picture to explore • Move the cursor while holding down the left mouse button – To see x, y, r, g, b • Click on zoom – To zoom in or out

Creating a Python Function Define a function Load the function Try the function

Rules for Python Functions • Use the keyword def to define a function def pick. And. Show(): my. File = pick. AFile() – Followed by the my. Picture = function name make. Picture(my. File) • Use parentheses after show(my. Picture) the name – Name parameters • End with a colon • Indent the body of the function

Using a Loop def decrease. Red(picture): for p in get. Pixels(picture): value = get. Red(p) set. Red(p, value * 0. 5)

More Picture Functions def greyscale(picture): for p in get. Pixels(picture): redness=get. Red(p) greenness=get. Green(p) blueness=get. Blue(p) luminance=(redness+blueness+greenness)/3 set. Color(p, make. Color(luminance, luminance)) def negate(picture): for px in get. Pixels(picture): red=get. Red(px) green=get. Green(px) blue=get. Blue(px) neg. Color=make. Color(255 -red, 255 -green, 255 blue) set. Color(px, neg. Color)

How Sound Works: • Sounds are waves of air pressure – Sound comes in cycles – The frequency of a wave is the number of cycles per second (cps), or Hertz – The amplitude is the maximum height of the wave

Volume and Pitch • Our perception of volume is related (logarithmically) to changes in amplitude – If the amplitude doubles, it’s about a 3 decibel (d. B) change. • Our perception of pitch is related (logarithmically) to changes in frequency – Higher frequencies are perceived as higher pitches – We can hear between 5 Hz and 20, 000 Hz (20 k. Hz) – A above middle C is 440 Hz

Digitizing Sound • We can estimate the sound curve – Analog-to-digital conversion (ADC) will give us the amplitude at an instant as a number: a sample – How many samples do we need? • 2 x max frequency • CD quality is 44, 100 samples per second

Digitizing sound in the computer • Each sample is stored as a number – 16 bits, 216 = 65, 536 combinations – But we want both positive and negative values • To indicate compressions and rarefactions. • Each sample can be between -32, 768 and 32, 767

Exploring a Sound Use Media. Tools to explore a Sound

Working with Sounds • We’ll use get. Samples to get all the sample objects out of a sound – Use get. Sample to get the value from a Sample obj • We can also get the value at any index with get. Sample. Value. At • Sounds also know their length (get. Length) and their sampling rate (get. Sampling. Rate) • Save a sound to a file using write. Sound. To(sound, ”file. wav”)

Decreasing the volume def decrease. Volume(sound): for sample in get. Samples(sound): value = get. Sample(sample) set. Sample(sample, value * 0. 5) This works just like decrease. Red, but decreasing the amplitude (volume) instead of the red color.

Multimedia Java CS 1 • We use Dr. Java – A free and easy development environment – With an interactions pane • We have created Java classes – For the students to use and modify • • • World Turtle Picture Sound File. Chooser

Java Syllabus • Introduction to Java – Math operators, printing results, data types, casting, relational operators, Strings, variables • Introduction to Programming – Creating and naming objects • Using a turtle and a world – Creating new Turtle methods • Draw simple shapes • Using parameters

Syllabus - Continued • Modifying Pictures using Loops – One-dimensional arrays – Use for-each, while, and for loops to – Increase/decrease colors, fake a sunset, lighten and darken, create a negative, and grayscale

Create and Show a Picture

Java Negate Method /** * Method to negate the picture */ public void negate() { Pixel[] pixel. Array = this. get. Pixels(); Pixel pixel. Obj = null; int red. Value, blue. Value, green. Value = 0; // loop through all the pixels for (int i = 0; i < pixel. Array. length; i++)

Java Negate Method - Cont { // get the current pixel. Obj = pixel. Array[i]; // get the values red. Value = pixel. Obj. get. Red(); green. Value = pixel. Obj. get. Green(); blue. Value = pixel. Obj. get. Blue(); // set the pixel's color pixel. Obj. set. Color( new Color(255 - red. Value, 255 - green. Value, 255 - blue. Value)); } }

Testing Negate

Syllabus - Continued • Modifying Pixels in a Matrix – Two-dimensional arrays – Nested loops – Copying, mirroring, blending, rotation, scaling

Syllabus - Continued • Conditionally Modifying Pixels – Replacing a color, reducing red-eye, edge detection, sepia-toned, posterize, highlight extremes, blurring, background subtractions, chromakey

Syllabus - Continued • Drawing on Pictures – Using existing Java classes • Inheritance • Interfaces – Drawing simple shapes, drawing text, general copy, general scale, shearing, gradient paint, general blending, clipping

Syllabus - Continued • Modifying all Samples in a Sound – – 1 D arrays Loops Conditional execution Change volume, normalizing a sound (make it as loud as possible), force to extremes

Syllabus - Continued • Modifying Samples using Ranges – Loops – Clipping, splicing, reversing, mirroring

Syllabus - Continued • Combining and Creating Sounds – Class and private methods – Composing sounds, blending sounds, changing frequencies, and creating echoes – MP 3 and MIDI

Syllabus - Continued • Creating Classes – Defining a class – Overloading constructors – Creating and initializing an array – Creating getters and setters – Creating a main method – Javadoc comments – Using Inheritance Turtle Confused. Turtle

Syllabus - Continued • Creating and Modifying Text – Reading from and writing to files • Handling Exceptions – Modifying programs – Getting text from networks – Creating random sentences – Using text to shift between media

Syllabus - Continued • Making Text for the Web – Throwing exceptions, Hash. Map, Generics, and Iterators – Create a web page from a directory – Create a web page from other web pages – Creating a web page from a database

Syllabus - Continued • Encoding, Manipulating, and Creating Movies – Frame-based animations with simple shapes and text – Special effects – fade out, fake sunset, and chromakey

Syllabus - Continued • Speed – What makes programs fast? • Compilers and Interpreters • Searching • Algorithms that can’t be written – What makes computers fast? • Clock rates, Storage, Display

Syllabus - Continued • Javascript – Syntax – User Interfaces – Multimedia

Multimedia CS 2 in Java • Driving question: “How did the wildebeests stampede in The Lion King? ”

CS 2 Ideas • Teach data structures using multimedia – Stacks and queues for simulations – Linked lists of pictures • Order in linked list is order in scene • Order in linked list is the layer the picture is in • Circular linked list for animation – Linked lists of sounds • Weave and repeat musical phrases – Trees of pictures • Scene graph – Trees of sounds • Different traversal gives different result

Examples of Student Work Soup. Audio Collage Canon. Linked. List of (MIDI) Music

Results • Python CS 1 is used for non-majors at Georgia Tech – Business students went from 49% to 88% pass rate – Similar results at Gainesville and U. Ill-Chicago • Java CS 1 is used in Georgia High Schools – Both in an introductory course and in Advanced Placement courses – Also used at Dennison and Duke • Java CS 2 is used at Georgia Tech – Students are transferring into computer science

Why Media Computation? • Works on nearly all computers – Unlike Alice which requires newer computers • Low cost – Unlike LEGO Mindstorms or other robots • Students can see (and hear) if their programs work • Computer Science concepts are introduced to solve problems – Looping instead of changing thousands of pixels by hand • Open-ended assignments allow students to be creative – Students can use their own media • Students can publicly share their media artifacts with others

What’s on the CD • CS 1 -Python materials – – Chapters from the Media Computation book in Python Course slides Free Jython Environment for Students (JES) Instructor Resources • CS 1 -Java materials – Chapters from the Media Computation book in Java – Course Slides – Java Classes

What’s on the CD - Continued • CS 2 -Java materials – Course notes – Course slides – Java Classes • Dr. Java – Free Development environment for Java • Mediasources – Royalty free pictures and sounds

Additional Resources • Web Site for Media Computation – http: //coweb. cc. gatech. edu/media. Comp-plan • CS 1 Python Book – Introduction to Computing and Programming in Python, A Multimedia Approach by Mark Guzdial, ISBN: 0131176552 • CS 1 Java Book – Introduction to Computing and Programming with Java, A Multimedia Approach by Mark Guzdial and Barbara Ericson, ISBN: 0131496980 • CS 2 Java Book – Coming in 2007

Funding Sources • National Science Foundation • Georgia Tech's College of Computing • Georgia’s Department of Education • Al West Fund • Toyota Foundation