Lingo Boingo Joining Forces to Promote Games for

Lingo. Boingo Joining Forces to Promote Games for Linguistic Research and Technology Development Christopher Cieri, James Fiumara supported by generous grants from the U. S. National Science Foundation (CRI CI-P 569801 and CRI CI-NEW 572166)

Background u Need l scant coverage of the world’s language l limits of current approaches u Inspirations –Libri. Vox, Zooniverse, GLG, etc. u Incentives l u information, entertainment, self-expression, socializing, developing skills, demonstrating competence, competition, status, prestige, recognition, access to (HLT) services, contributing to a greater cause or good Workforces & X-sourcing l recruiting, task scaling, assessing skills, multiple annotations, comparing u Workflows & Post-processing u NIEUW & its role in the effort l infrastructure l activities l portals: games, citizen science, language professionals Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 2

Inspiration: Great Language Game >16 M judgments in 9 months >38 M judgements extrapolated probably many more but only confusability data, no new annotations Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 3

Name that Language! Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 4

Game Parameters u u Game Play: using bonus round to collect novel annotations l # moves, levels, ‘lives’ l reward for correct, penalty for incorrect - normally and in bonus round l frequency of bonus rounds l skip or answer before clip complete l # languages, # distractors, closed set Data l l language of clips relative to player’s n location n native language n performance language of distractors relative to n all above n distance from target language • phonemically • in terms of family tree Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 5

Others: Language Squad seems easier than Great Language Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 6

Others: Ling. Your. Language seems much more difficult than Great Language Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 7

Lingo. Boingo Portal u Portal - links u Authentication u Back-end u Publicity l Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan analysis 8

Publicity: Planned hat tip U. Essex Events u u u November: sites snapshot current # players, # annotations December: LDC newsletter, social media; cross promotion among sites January: Linguist. List mailer (includes single post to FB, Twitter, G+) Measures u Games u Items u Annotations Per Item u Countries u Lifetime Judgements u February: Sci. Starter u Average Judgements per Player u March: Facebook direct u Average Lifetime Play April: Philadelphia Science Festival (55, 000 attendees to Carnival) u Monthly Active Users u Retention (months/weeks) u u May: Twitter direct u Throughput (hours/days) u June: Google Ad. Words u User Performance (precision, recall, f) u System Performance (precision, recall, f) l Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan with and without voting 9

Publicity: Actual u December l u u n 12/6 Facebook and Twitter pages n 12/15 Newsletter n 12/18 “What’s New” on homepage Linguist. List n 1/22 Mailer n Facebook post n Twitter post n G+ post April l Sci. Starter n l Philadelphia Science Fair n u 4/4 project added 4/28 May l Sci. Starter n 5/15 promotion campaign to start February l u LDC January l u Sci. Starter discussion started March l Sci. Starter discussion continues Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 10

Sessions/Day & Outgoing Clicks LDC Linguist. List Sci. Starter PSF Correlation with sessions received at: Tile Attack=. 96 Zombilingo=. 94 Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 11

Geographical Distribution Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 12

Findings & Future u u u Findings l Still ‘intra-research-community’ l Not everyone who visits Lingo. Boingo plays a game immediately l Individual publicity events affect games differently. l Use decreases during weekends l Entropy but some cumulative effect apparent To join: l game l resulting data shared with research community l track impact of publicity Thanks to: l Massimo Poesio, Udo Kruschwitz, Jon Chamberlain, Chris Madge; Karën Fort, Bruno Guillaume; Mathieu Lafourcade for their collaboration Games 4 NLP Workshop, LREC 2018, 7 May, Miyazaki, Japan 13