Sound in Java Multimedia Systems Module 1 Lesson

Sound in Java Multimedia Systems (Module 1 Lesson 4) Summary: r Sound API Basics r Playing Audio m m In Memory (bounded) Streamed (unbounded) r Recording Sound Sources: r Chapter 18 of this book: Beginning Java 2 – JDK 1. 3 Version by Ivor Horton r Link to examples is available on the class website r MIDI 1

Java Sound Preliminaries File Formats Supported Frames and Frame Rates r. au or. snd : usually stores r Sample Frame 8 -bit m-law encoded m Stores all the samples taken samples, but can also store 8 at an instant of time or 16 bit linear samples m # of bytes in a frame = # of r. aif : usually stores 8 or bytes in a sample X number of 16 bit linear encoded channels samples r Frame Rate r. wav : Can store 8 or 16 bit m The number of frames per samples using linear or m-law second of sound encoded samples m In most cases frame rate is r. midi : follows the midi same as sample rate data format m Note: The file header indicates the actual format In compressed sound, the frame rate will be less than sample rate. 2

Simple Sound Output Play. It. java import java. applet. *; import javax. swing. *; import java. awt. event. *; public class Play. It extends JApplet { Audio. Clip clip; // The sound clip JButton button; final String play = "PLAY"; final String stop = "STOP"; public void init() { // Get the sound clip String file. Name = get. Parameter("clip"); // Get the file name clip = get. Audio. Clip(get. Document. Base(), file. Name); // Create the clip button. add. Action. Listener(new Action. Listener() { public void action. Performed(Action. Event e) { if(e. get. Action. Command(). equals(play)) { clip. loop(); button. set. Text(stop); } else { clip. stop(); button. set. Text(play); } } } ); get. Content. Pane(). add(button); } // Rest of the applet initialization. . . button = new JButton(play); Playit. html <APPLET CODE =“Play. It. class” CODEBASE = “. ” WIDTH = 300 Height = 50 clip = “my. Clip. wav” > </APPLET> 3

Sound in Applications Similar to an Applet r The Applet class defines a static method, new. Audio. Clip()that retrieves an audio clip from a URL and returns a reference type Audio. Clip that encapsulates the audio clip. m This method being static, you don’t have to have an applet object to call it. It can be called from an application as well, like so: Audio. Clip clip = Applet. new. Audio. Clip(URL location) m Take a look at example Play. Sounds. java in the examples directory 4

Java Sound API System Resources r Audio Line: Any resource that is a source or destination of sampled sound data m m m A line can encapsulate several channels Example: input/output ports on the sound card Lines have controls (gain and pan control) r Mixer: Receives input from one or more source data lines and outputs the result of combining the input to an output line called a target data line Other Sound Sources r A file or more generally a URL Terminology: r A source data line is a source for a mixer, not a source for you; you write to it r A target data line is the output from the mixer; your read from it Source Data Lines Target Data Lines Sound i/p MIXER Sound o/p 5

Java Sound API (…contd) Packages: Audio. Stream. Input class r javax. sound. sampled r Represents a stream that is r javax. sound. midi r javax. sound. sampled. spi r javax. sound. sampled. midi The Audio. System class r Establish whether a particular resource is available r Get a ref. to the object that encapsulates the resource r Call methods to operate the resource a source of sampled sound data with a specific format r You can create an Audio. Stream. Input object from a local sound file, from another input stream or a URL r You can m m m Read data from it Write its contents to an output stream Convert its format 6

Sampled Sound Interfaces Line Mixer Source. Data. Line Target. Data. Line Port Clip 7

Resource Descriptor Classes The Line, Data. Line, Mixer and Port interface definitions each include an inner class with the name Info. Objects of these class types encapsulate data specifying an object of the corresponding type Specifies a Line object by its Class Line. Info Mixer. Info Specifies a Mixer object by its name, vendor version and description Data. Line. Info Specifies a Data. Line by the audio formats to be supported, the buffer size limits, and the Class of the data line Port. Info Specifies a Port object by its Class, its name, and whether it is a source or target line 8

Why Descriptor Class? In order to answer that question, we have to look at the steps involved in playing audio: 1. Create an instance of a Line that specifies the format of the sound data you wish to play (use descriptor class). 2. Get a reference to a line (a Data. Line, Port, Mixer or Clip) that supports this format 1. 3. Create an Audio Stream that encapsulates the sound data (file, URL, another stream) 1. 4. 5. May check if supported before requesting reference Extract the format from the Audio Stream Tie the source of the sound data to the reference (line) that will play it. I. e. , open the source Play it; loop; goto; quit. 9

Playing a Clip vs. Stream Audio. Input. Stream source = Audio. System. get. Audio. Input. Stream(file); // Step 3. Data. Line. Info clip. Info = new Data. Line. Info(Clip. class, source. get. Format()); // Step 1. if(Audio. System. is. Line. Supported(clip. Info)) { Clip new. Clip = (Clip)Audio. System. get. Line(clip. Info); // Step 2. new. Clip. open(source); // Step 4. } clip. loop(clip. LOOP_CONTINUOUSLY); // loop clip. stop(); // stop clip. set. Frame. Position(0); Clip. close(); Audio. Input. Stream new. Source = Audio. System. get. Audio. Input. Stream(file); // Step 3. Audio. Format format = new. Source. get. Format(); Data. Line. Info source. Info = new Data. Line. Info(Source. Data. Line. class, format); // Step 1. if(Audio. System. is. Line. Supported(source. Info)) { src. Line = (Source. Data. Line)Audio. System. get. Line(sou rce. Info); // Step 2. buffer. Size = (int)(format. get. Frame. Size()*format. get. F rame. Rate()/2. 0 f); sound. Data = new byte[buffer. Size]; src. Line. open(format, buffer. Size); //4. } while(playing) { byte. Count = source. read(sound. Data, 0, sound. Data. length); // Read the stream if(byte. Count == -1) { source. Line. drain(); // rest of buffer playing = false; break; } source. Line. write(sound. Data, 0, byte. Count); // Write the array to the line 10 }

MIDI in Java. Sound r Data is a MIDI file is a series of commands that defines a piece of music r Up to 16 MIDI channels are available (each instrument uses one channel) r A MIDI Synthesizer reproduces(synthesizes) sounds in response to MIDI commands m m H/W part of the sound card S/W as in Java. Sound r A sequencer is a device that processes a MIDI sequence in order to play it on a synthesizer, or possible to edit it. m H/W or S/W A device conforming to the General MIDI spec. must provide: r A min. of 128 preset instruments + 47 percussive sounds r A min. of 16 simultaneous timbres (instruments) r A min. of 24 simultaneous voices, where each voice is a note of given velocity (loudness) for any of the available instruments and percussive sounds r 16 midi channels, where each channel is polyphonic(can play multiple simultaneous voices). Percussive sounds are always on channel 10 11

MIDI Resources in Java. Sound Declares basic operations for a MIDI device. Inner class, Mid. Device. Info can be used for specifying a device Midi. Device Declares operations specific to a sequencer. Innerclass Sequencer. Sync. Mode used for synchronizing with another device Sequencer A sequencer object can play a sequence, which you can construct from a MIDI file Declares operations specific to a synthesizer Synthesizer A synthesizer object encapsumates a h/w or s/w MIDI synthesizer 12

Playing a MIDI file To play a MIDI file, you don’t need to access a synthesizer directly. All you need is a Sequencer reference and an object encapsulating the sequence that you want to play. Steps: 1. sequencer = Midi. System. get. Sequencer(); // Get a sequencer 2. sequencer. open(); sequence = Midi. System. get. Sequence(midi. File) // Encapsulate the midi src (file here; URL possible) in a sequence obj. sequencer. set. Sequence(sequence); // Hand the sequence over to the sequencer. start(); // Play it. Stop it: sequencer. stop() 3. 4. 5. 13