1 001 001 Intro To Computation and Engineering

1. 00/1. 001 – Intro To Computation and Engineering Problem Solving Fall 2002 Media in Java®

Topics • The AWT, Java® 2 D API and the newer Java® Media Framework provide tools for working with diverse media. – structure of digital media – the AWT Image class and its related methods – the Media. Tracker class – the Java ® Media Framework structure – use of the JMF for video and audio playing – media synchronization

Digital Media • Includes text, video, audio, animation, images and vector graphics • video and audio have temporal dimension • files are often large and compressed • diverse file formats • latency and network throughput are often issues

The AWT and JMF • The AWT supports text, limited drawn graphics and text. • Java® 1. 2 provides much improved graphics and text support, and additional image processing features • the Java® Media Framework is an additional package that provides support for temporal media-audio and video • JMF features include synchronization and implementation using native platform features

Images in the AWT • Image class supports the "push model". Images are "pushed" to an Image. Consumer. Works well for applets and other Web applications. • A digital image (once loaded) is a rectangular collection of pixels, where each pixel represents a color • Java® AWT has an Image class for loading and displaying images • AWT can load image from file or URL over the Web • The Java® 2 D API has Buffered. Image class that has much wider range of capabilities

Image in an Applet • Simplest case is loading an image into an application • construct an Image object with code such as: Image im = new Image(“my. Image. File. gif”); • display image in the paint. Component() method of Component object using draw. Image() method: boolean b draw. Image(Image im, int x, int y, Image. Observer iob) returns true if Image is fully loaded, false otherwise

Image. Observer interface • Images take a long time to load, and may load incrementally over time • Image. Observer is an interface that has methods for handling notification of state of image loading. It can use this for redisplay as needed. • JFrame and Applet both implement Image. Observer interface

Drawing an Image import java. awt. *; import javax. swing. *; public class Image. Panel extends JPanel { private Image im; public Image. Panel(Image i) { im = i; } public void paint. Component(Graphics g) { super. paint. Component(g); if(im == null) g. draw. String("Image object null", 30); else g. draw. Image(im, 10, this); } } // end of Image. Panel class

Other Image loading methods public Image get. Image(URL u) loads image from Web from URL u. URL is a class in java. net package. Its simplest constructor is URL u = new URL(String s) public Image get. Image(URL u, String name)-u is the base location on a Web server; name is the file name

Media. Tracker class • Sometimes you want to (or have to wait) for image to load fully before proceeding • Media. Tracker class allows you to register images and wait until they are fully loaded • supports waiting for groups of images • you register images with Media. Tracker object • Media. Tracker can establish priorities for image loading

Media. Tracker Constructor and Methods Media. Tracker(Component m) -m is the object where the images are to be drawn void add. Image(Image im , int id)-im is the image to be tracked, id is the group of objects with same id void wait. For. ID(int id) throws Interrupted. Exception void wait. For. All() throws Interrupted. Exception void wait. For. ID(int id, long ms) throws Interrupted. Exception void wait. For. All(long ms) throws Interrupted. Exception boolean check. ID(int id) boolean check. All()

Media. Tracker Use Media. Tracker i. Track = new Media. Tracker(this); Image t. Image = new Image(“cecif 2. jpg”); i. Track. add. Image(t. Image, 1); try { i. Track. wait. For. ID(1); } catch(Interrupted. Exception e) { System. err. println(“Error in Image loading”); }

import java. awt. *; import javax. swing. *; public class Image. Display extends JFrame { private Image test. Image; private Image. Panel i. Panel; public static void main(String[] args) { new Image. Display(args[0]); } public Image. Display(String image. File) { set. Title("Simple Image Displayer"); Media. Tracker i. Track = new Media. Tracker(this); test. Image = Toolkit. get. Default. Toolkit(). get. Image(image. File); i. Track. add. Image(test. Image, 1); try { i. Track. wait. For. ID(1); } catch(Interrupted. Exception e) { System. out. println("Exception in loading image(s). "); System. exit(0); }

i. Panel = new Image. Panel(test. Image); get. Content. Pane(). add(i. Panel); set. Size(test. Image. get. Width(this)+50, test. Image. get. Height(this)+50); set. Default. Close. Operation(JFrame. EXIT_ON_CLOSE); set. Visible(true); } } // end of Image. Display

Java. R 2 D API • Java® 2 D API provide "immediate mode“ model for images. the Buffered. Image class is central to this. • Buffered. Image class allows much greater programmatic control than AWT's Image class • Buffered. Image supports affine transformations, image filtering, multiple color models, compositing, multiple compression methods

import java. awt. *; import javax. swing. *; import java. io. *; import com. sun. image. codec. jpeg. *; // JPEG Decoder public class Buffered. Image. Display extends JFrame { private Image test. Image; private Image. Panel i. Panel; public static void main(String[] args) { new Buffered. Image. Display(args[0]); }

public Buffered. Image. Display(String image. File) { set. Title("Simple Image Displayer"); try { Input. Stream in. Stream = new File. Input. Stream(image. File); JPEGImage. Decoder de = JPEGCodec. create. JPEGDecoder(in. Stream); test. Image = de. decode. As. Buffered. Image(); } catch(IOException e) { System. out. println("Exception in loading image "+image. File); System. exit(0); } i. Panel = new Image. Panel(test. Image); get. Content. Pane(). add(i. Panel); set. Size(test. Image. get. Width(this)+50, test. Image. get. Height(this)+50); set. Default. Close. Operation(JFrame. EXIT_ON_CLOSE); set. Visible(true); } } // end of Image. Display

Inside Buffered. Image Class • Buffered. Image object has two major parts: – Raster-storage for pixels, each representing small region of image – Color. Model-interpretation of pixel value (maps pixel contents to color components) • Buffered. Image class provides wide range of options for packing pixels, color models, indexed colors

Common Color Spaces • RGB-Red, Green Blue, often with an Alpha component for opacity • CMYK – Cyan, Magenta, Yellow and Black • Gray. Scale There are usually measured with respect to standard references for color

Buffered Operations • There are standard filters for operating on Buffered. Images that implement the Buffered. Image. Op interface • Basic coding pattern: – create source Buffered. Image object – Use a "factory method" to create an appropriate converter that implements Buffered. Image. Op interface – run converter object's filter method, as in Buffered. Image result = converter. filter(source, null); // null can be the destination image

Image Sharpening Example import java. awt. *; import java. awt. image. *; import javax. swing. *; import java. io. *; import com. sun. image. codec. jpeg. *; // JPEG Decoder public class Sharpen. Op extends JFrame { private Buffered. Image test. Image; private Image. Panel i. Panel; private Buffered. Image conv. Image; public static void main(String[ ] args) { new Sharpen. Op(args[0]); }

public Sharpen. Op(String image. File) { set. Title("Sharpen Image Displayer"); try { Input. Stream in. Stream = new File. Input. Stream(image. File); JPEGImage. Decoder de = JPEGCodec. create. JPEGDecoder(in. Stream); test. Image = de. decode. As. Buffered. Image(); } catch(IOException e) { System. out. println("Exception in loading image "+image. File); System. exit(0); }

float kern[ ] = {-1 f, 9 f, -1 f, -1 f}; Kernel s. Kernel = new Kernel(3, 3, kern); Convolve. Op sharp = new Convolve. Op(s. Kernel); conv. Image = sharp. filter(test. Image, null); i. Panel = new Image. Panel(conv. Image); get. Content. Pane(). add(i. Panel); set. Size(conv. Image. get. Width(this)+50, conv. Image. get. Height(this)+50); set. Default. Close. Operation(JFrame. EXIT_ON_CLOSE); set. Visible(true); } } // end of Sharpen. Op

Java® Media Framework Goals • make Java® applications using media portable • relieve programmer of details of diverse media types • provide means for using native decompression codes in Java® applications • tools for synchronizing two or more temporal media • allow Java® programmer to treat different media types uniformly independent of format and network • provide useful defaults for media players and still allow customization when desired

JMF Concepts • Data. Source -source of media content--file server, etc. • Two types are Push. Data. Source and Pull. Data. Source • Example URL m. URL = new URL(“http: //wwwceci. mit. edu/seg 6. mpg”); Data. Source my. Source = new Data. Source(m. URL)

Player Interface • Player-an interface for a player of a media type. Classes meeting this interface will typically involve native code. Implementations of Player can be “plug ins”. • Concept is to allow many different groups build Player implementations.

Manager class • A “factory” that can create a player appropriate to particular Data. Source. • Manager can locate Player classes if available and provide them to applications public static Player create. Player (Data. Source source) throws IOException, No. Player. Exception public static Player create. Player(URL source. URL) throws IOException, No. Player. Exception

Sample Manager use // assume media. URL is a valid URL object try { player = Manager. create. Player(media. URL); } catch(No. Player. Exception e) { System. err. println("No player for URL"); return; } catch(IOException e) { System. err. println("IO Exception encountered"); return; }

Player States • unrealized -Player has been created but doesn't know anything about its media except the URL • realizing -Player is in process of acquiring whatever resouces it needs • realized -Player finished realizing and knows about its resources and media. It now knows how to display itself. • prefetching -Player preparing to present media • prefetched -Player done preparing to present media • started -clock of Player is running

State Diagram for Player Source: Java. TM Media Player White Paper, Sun web site

Player Components • public Component get. Control. Panel. Component() – returns a panel with controls for media (start, stop, etc. ) • public Component get. Visual. Component()-returns a Component in which media can be displayed • public Component get. Gain. Component() -returns a Component with audio volume controls

Player Events • Player can generate a large number of different Event types specific to JMF • Controller. Event is most useful • Player can register Controller. Listener objects for notification. Uses AWT event model • Controller. Listener interface must have controller. Update(Controller. Event e) methods

JMF Event Types

Sample JMF Usage • Create Frame for video player to display in • create URL for video file • Use Manager class to create appropriate Player for media • handle event where Player is fully realized, and add visual and control Component of Player to Frame • start the Player by calling player. start() method

import java. awt. *; // import Java AWT package import java. awt. event. *; // import Java event package import java. net. *; // import network package import java. io. *; // import Java IO package import javax. media. *; // import JMF package public class Video. Player implements Controller. Listener { Player player = null; Component visual. Component, control. Component; Container m_container; public static void main (String args[]) { String media. File = args[args. length-1]; Frame fr = new Destroyable. Frame("Simple Video Player"); fr. set. Size(428, 398); fr. set. Visible(true); Video. Player player = new Video. Player(media. File, fr); } // end of main method for Video. Player class

// constructor method for Video. Player class public Video. Player(String media. File, Container container) { URL media. URL = null; m_container = container; m_container. set. Layout(new Border. Layout()); try { media. URL = new URL(media. File); player = Manager. create. Player(media. URL); player. add. Controller. Listener(this); player. realize(); } catch (Malformed. URLException e) { System. err. println("Invalid media file URL!"); return; } catch(No. Player. Exception e) { System. err. println("Unable to find a player for "+media. URL); return; } catch(IOException e) { System. err. println(”IO Exception encountered"); return; } } // end of Video. Player constructor method

// When player is realized, get its visual and control Components and // put them into the container and start() the complete player. public void controller. Update(Controller. Event event) { if(event instanceof Realize. Complete. Event){ if((visual. Component = player. get. Visual. Component()) != null) m_container. add("Center", visual. Component); if ((control. Component = player. get. Control. Panel. Component()) != null) m_container. add("South", control. Component); m_container. validate(); // ensure layout is up to date m_container. repaint(); // ensure that anything drawn painted player. start(); // start the player } else if (event instanceof Controller. Error. Event){ System. err. println("*** Controller. Error. Event *** " + ((Controller. Error. Event)event). get. Message()); } } // end of controller. Update method } // end of Video. Player class Java® is a trademark or registered trademark of Sun Microsystems, Inc. in the United States and other countries.