Optical Music Recognition and Data ImportExport Music 253

Optical Music Recognition and Data Import/Export Music 253/ CS 275 A Eleanor Selfridge-Field

Optical Music Recognition (OMR) History of efforts from c. 1968 • CCARH survey in 1993 -4: 37 projects, 7 responses Why is optical recognition difficult? • Semantic meaning of many objects depends on graphical context more than shape Sources and their legibility: • • Manuscripts: very irregular Out-of-copyright prints: images often deteriorated In-copyright prints: not legal to copy Errors in source Biggest problems for OMR developers • • Superimposition of objects in 2 D image Constraints imposed by output formats Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 2

Basic problems in optical data acquisition • Image is crooked Mus 253/CS 275 A • Elements of layout unconventional 2019 Eleanor Selfridge-Field 3

How does OMR work? • Separation of lines and other Captured: matternotes, rests • Isolation of objects • Recognition of objects • Export to a format for • • storage printing sound data interchange Missed: slurs, pedal marks Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 4

Why is OMR difficult? Problems of image quality: • Ideally • Staff lines are straight • Spacing is uniform • The scanned material is clean (unspotted) Slurs are symmetrical Beams are parallel All lines are unbroken • Problems of graphical context • • • Reality is different! • Symbols affecting interpretation of pitch • • • Symbols affect interpretation of duration • • Meter signatures Tempo indicators Fermatas Symbols relating to dynamics and technique • • • Mus 253/CS 275 A Key signatures Octave alterations 2019 Eleanor Selfridge-Field Dynamics marks Repetition of note-groups , of sections Instrumental technique 5

More difficulties Multiple configurations for same objects Methods of evaluation and control • Musical accuracy? • Handicaps for post-processing • Controls for input quality • Comparison of output formats • Weighing speed against accuracy and usability Haydn symphony n. 1 capture Fujinaga et al. Mus 253/CS 275 A Input Capture format Post. Processing Carter, scan 00: 20 SCORE 9: 20 CCARH, 2 -stage entry 2: 30 + 7: 05 Muse. Data 00: 15 2019 Eleanor Selfridge-Field 6

Samples from Library of Congress site Random material from loc. gov Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 7

Close-up views of conventionally typeset music: Line recognition Surface imperfections Mus 253/CS 275 A Surface imperfections 2019 Eleanor Selfridge-Field 8

Close-up views (2): object recognition Missing contextual information Mus 253/CS 275 A Graphic imperfections 2019 Eleanor Selfridge-Field 9

Close-up views (3): 2 D ambiguities Dirt Mus 253/CS 275 A Variable appearance of equivalent objects 2019 Eleanor Selfridge-Field 10

Close-up views (4): ambiguities of placement Touching objects Mus 253/CS 275 A Unconventional presentations 2019 Eleanor Selfridge-Field 11

Sharp. Eye: File operations Comes from Shetland Islands Source code available Exports to Music. XML Four-step process • • Capture a page image View the auto-image Correct the image Save/export the result Vis-à-vis Muse. Data: • • Mus 253/CS 275 A 2019 Eleanor Selfridge-Field SE: score-based MD: part-based 12

Sharp. Eye: Raw Capture Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 13

Sharp. Eye: Correcting the interpretation • Edit mode: • • Captured image below Interpreted image above Live object in red Available symbols in red Step 1: 1: Selectionaaportionthe scoretotoedit Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 14

Sharp. Eye: Scrolling view Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 15

Sharp. Eye: System edits Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 16

Sharp. Eye: Data-interchange options Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 17

Other OMR Software • Neuratron Photo. Score: http: //www. neuratron. com/photoscore. htm [Sibelius] • Smart. Score: http: //www. musitek. com/ Capella. Scan Photo. Score Smart. Score Mus 253/CS 275 A 2019 Eleanor Selfridge-Field 18

Important questions about OMR software • What does “accuracy” mean? • Text recognition optimal error rate: 40/2000 chars • What output formats are available? • MIDI-level features only? • Graphical position? • Markup? • What kinds of errors? • Global variables? • Local events? • Non-MIDI objects Mus 253/CS 275 A 2016 Eleanor. Selfridge-Field 2019 20 19