Sound you tube Some from Heim Chap 13
Sound you tube Some from Heim Chap 13 The sound files are not included– most should work on windows 7 machines
Learning outcomes • Describe the basics of human hearing • Explain the difference between visual and auditory interaction • Describe the classes and subclasses of sound output and the attributes of each • Describe the classes and subclass of sound input and recognition and attributes of each 2
Hearing • Provides information about environment: distances, directions, objects etc. • outer ear • middle ear • inner ear – protects inner and amplifies sound – transmits sound waves as vibrations to inner ear – chemical transmitters are released and cause impulses in auditory nerve • Sound • pitch • loudness • timbre – sound frequency – amplitude – type or quality the human 1 3 • Physical apparatus:
Hearing (cont) • Humans can hear frequencies from 20 Hz to 15 k. Hz • Auditory system filters sounds • can attend to sounds over background noise. • for example, the cocktail party phenomenon. • Hearing aids disrupt this filtering the human 1 4 • less accurate distinguishing high frequencies than low. • Higher frequencies disappear as you get older
What if…. • You are in a noisy environment • Night clubbing • Your hearing is below average • You are deaf the human 1 5 • Phone call/ text message?
Sound versus Visual Sound exists in time and over space, vision exists in space and over time. (Gaver, 1989) - Sound is only there when it is playing/made - Vision is there until it is replaced 6
Sound Interaction • Computer Output/Generation (Human Input) • Non speech • Music • Audio Icons and Earcons • Speech • Computer Input/Recognition • Speech • Non speech • Environmental • Music 7
Computer Output: Music • Can be pre-recorded or generated • Movies • Games • Immersive experiences 8
Generating music • Exciting area for artists • Everything from pseudo real to completely abstract • There are Jazz music generators that only skilled people can differentiate from actual musicians. • Serato – dj software (www. serato. com) • Auckland company doing fantastic things • Several UOA grads there 9
Auditory Icons and Earcons • The difference between these two is subtle • ‘Natural’ sounds • ‘Artificial’ sounds (generated) Windows hardware fail insert remove 10
Auditory Icons and Earcons • Redundant Encoding • It aids memory by adding additional associations. • Can alert without interrupting • An alterative communications channel. • Positive/Negative Feedback • Auditory alarms might be crucial to the safe operation of computer-operated machinery or mission-critical environments • Too many alarms • Annoying • Ignored 11
Using Sound in Interaction Design • Learnability of the mapping between the icon and the object represented • “Oink” and “bow wow” have high articulatory directness • A swishing sound accompanying a paintbrush tool also has high articulatory directness • A system beep carries is an abstract representation 12
Innate human understanding • Volume <quiet ----------------------------------loud> • Whisper • Shout • Pitch • Baby crying • Reassurance • Authoritative • Rhythm • Urgency • Mood • Length • Start, stop, duration 13
Can you remember earcons? • How many? • How often do you hear them? • Can you intuitively tell what these mean? On Off Sleep 14 Misrecognized Disambiguate
Speech Output • Eyes free operation • Alternative output channel • Good for checking your essays • Navigation is hard • Back tracking, • Finding location of particular thing 15
Speech Output • Recorded • Menu choices for telephone systems • Books or other multimedia experiences • Generated • Good voice synthesisers built in to op systems • Windows navigator • Can give pronunciation rules • Still sound a little artificial 16
Sound Input • Speech • Environmental • Music 17
Speech Recognition • Two distinct applications: • Transaction • Transcription • Transaction • Telephone menu systems • Choose from a limited number of options, works ok • Automatic speech recognition (ASR) • Built into operating systems • Siri (i. Phone) and Android are ~~ usable • This is a triumph of Artificial Intelligence • Very difficult, ongoing research problem 18
Searching Speech and Audio • Sound files do not afford easy opportunities for indexing and searching • Speech translation can be used to transcribe speech files and create transcripts that can be searched like any other text file • So long as recognition accuracy is ok, which it isn't at the moment • Tune identification apps • hum a bit of the tune and it tells you what it is! • Easier than translation 19
Summary • Describe the basics of human hearing • Explain the difference between visual and auditory interaction • Sound is transitory • Describe the classes and subclasses of sound output and the attributes of each • Non speech • Music • Earcons • Speech • Describe the classes and subclass of sound input and recognition and attributes of each • Speech • Transaction • Transcription • Sound is more complex than colour/light 20
- Slides: 20