Supporting Collaborative Writing with Microtasks Jaime Teevan Shamsi

← Supporting Collaborative Writing with Microtasks Jaime Teevan, Shamsi Iqbal, Curtis von Veh Microsoft Research →

Nebeling, To, Guo, de Freitas, Teevan, Dow & Bigham. Wear. Write: Crowd-assisted writing from smartwatches. CHI 2016. Kittur, Smus, Khamkar & Kraut. Crowd. Forge: Crowdsourcing complex work. UIST 2011. Agapie, Teevan & Monroy-Hernández. Crowdsourcing in the field: A case study using local crowds for event reporting. HCOMP 2015. Luther, Hahn, Dow & Kittur. Crowdlines: Supporting Synthesis of Diverse Information Sources through Crowdsourced Outlines. HCOMP 2015. Hahn, Chang, Kim & Kittur. The Knowledge Accelerator: Big picture thinking in small pieces. CHI 2016. Kim, Cheng & Bernstein. Ensemble: Exploring complementary strengths of leaders and crowds in creative collaboration. CSCW 2014. Bernstein, Little, Miller, Hartmann, Ackerman, Karger, Crowell & Panovich. Soylent: A word processor with a crowd inside. UIST 2010.

② ① Collect content ① ② Organize content ③ Turn content into writing ③ ① ② ③ Sadauskas, Byrne & Atkinson. Mining memories: Designing a platform to support social media based writing. CHI 2015.

① Collect Content The Micro. Writer breaks writing into microtasks Microtasks can be shared with collaborators Microtasks can be done while mobile Collaborative writing typically requires coordination Collaborators can be known or crowd workers People have spare time when mobile Structure turns big tasks into series of small microtasks Microtasks make it easy to get started

① Collect Content

② ① Collect content ① ② Organize content ③ Turn content into writing ③ ① ① ② ③ Chilton, Little, Edge, Weld & Landay. Cascade: Crowdsourcing taxonomy creation. CHI 2013.

② Organize Content

② Organize Content collab microtask mobile tasks Microtasks can be shared with collaborators

② Organize Content collab microtask mobile The Micro. Writer breaks writing into microtasks Microtasks can be shared with collaborators Microtasks can be done while mobile Collaborative writing typically requires coordination Collaborators can be known or crowd workers People have spare time when mobile Structure turns big tasks into series of small microtasks Microtasks make it easy to get started

② ① Collect content ① ② Organize content ③ Turn content into writing ③ ① ② ③

③ Turn Content into Writing ② collab ① microtask ③ mobile The Micro. Writer breaks writing into microtasks Microtasks can be shared with collaborators Microtasks can be done while mobile Collaborative writing typically requires coordination Collaborators can be known or crowd workers People have spare time when mobile Structure turns big tasks into series of small microtasks Microtasks make it easy to get started

③ Turn Content into Writing • microtask Microd • The Micro. Writer breaks writing into microtasks • Structure turns big tasks into series of small microtasks • Microtasks make it easy to get started collab • Collab • Microtasks can be shared with collaborators • Collaborative writing typically requires coordination • Collaborators can be known or crowd workers mobile • Mobile • Microtasks can be done while mobile • People have spare time when mobile

③ Turn Content into Writing

③ Turn Content into Writing collab • Microtasks can be shared with collaborators • Collaborative writing typically requires coordination • Collaborators can be known or crowd workers Collaborative writing typically requires coordination. However, microtasks are easy to share with collaborators without the need for coordination. The collaborators can be known colleagues, or paid crowd workers.

③ Turn Content into Writing • Complete output: Structure makes it possible to turn big tasks into a series of smaller microtasks. For example, the Micro. Writer breaks writing into microtasks. These microtasks make the larger task easier to start. Collaborative writing typically requires coordination. However, microtasks are easy to share with collaborators without the need for coordination. The collaborators can be known colleagues, or paid crowd workers. People have spare time when mobile, and these micromoments are ideal for doing microtasks.

② ① Collect content ① ② Organize content ③ Turn content into writing ③ Micro. Writer ① ② ③ ③

Collaborative Writing with the Micro. Writer • Six groups of active collaborators • 2 to 5 people in each group • Total number of people: 19 Coordination Location Collect Individual Co-located Initial Individual Co-located Merge Group Co-located Finalize Individual Co-located Individual Remote Organize Stage Write

Writing with the Micro. Writer • Self-motivated report topics • • System description Overview of the group Research plan A rebuttal • Time to write: 1 hour • Length: 723 words on average • Example: See Discussion Section Ideas 30 45 23 Labels 28 13 8 Groups 12 5 5 Words 752 605 234 89 37 21 57 15 13 22 7 4 1160 801 789

Key Micro. Writer Opportunities • Easy to get started • “It was a relatively fast way to divide the work and produce a great starting point… I think we’ll actually use this. . ” • Everyone got a voice • “I felt the process enabled us all to contribute significantly. ” • Intertwined contributions • “Typically. . one of us would write a full draft and circulate. The tool changes this up a bit by producing an initial draft that is drawn up collaboratively. ” Content: • Microtasks can be shared with collaborators • Collaborative writing typically requires coordination • Collaborators can be known or crowd workers Collaborative writing typically requires coordination. However, microtasks are easy to share with collaborators without the need for coordination. Collaborators can be known colleagues, or paid crowd workers.

Key Micro. Writer Challenges • Writing via microtasks • Bottom-up organization unfamiliar • Tried to consider all of the implications of each micro-action • Label merging least favorite stage • People wanted to go back and correct errors • Opportunity: Liked the unexpected connections • Coordination and collaboration • Individual nature of microtasks discouraged explicit collaboration • Hard to understand the content other people entered • Opportunity: Intentional task allocation

Summary • Possible to decompose writing into a series of microtasks • • Simple writing workflow creates reasonable output Microtasks make it easy to start writing Using a structured writing process is unfamiliar Bottom-up organization creates unexpected connections • Writing microtasks can be shared with known collaborators • Reduces the need for collaborators to coordinate work • Everyone gets a voice and helps produce the final product • The need to develop a shared understanding remains

← Questions? More at #CHI 16 about microtasked writing: Cai et al. Chain reactions: The impact of order on microtask chains. Today at 4: 30 pm (next session) in Room LL 20 B. Nebeling et al. Wear. Write: Crowd-assisted writing from smartwatches. Tomorrow at 11: 30 am in Room LL 21 B. →