Crowdsourcing Quality Management and other stories Panos Ipeirotis
- Slides: 26
Crowdsourcing Quality Management and other stories Panos Ipeirotis New York University & Tagasauris
Amazon Mechanical Turk
Tagging patent pending
But what if MTurk workers submit incorrect tags? patent pending
Voting
Voting But what if MTurk workers also vote incorrectly?
Voting But what if MTurk workers also vote incorrectly? Redundancy: 5+ votes per tag
But my task is not just right or wrong… Describe this Creation HIT (e. g. describe the image) Improve HIT Compare HIT (voting) (e. g. improve description) Which is better?
version 1: A parial view of a pocket calculator together with some coins and a pen. version 2: A view of personal items a calculator, and some gold and copper coins, and a round tip pen, these are all pocket and wallet sized item used for business, writting, calculating prices or solving math problems and purchasing items. version 3: A close-up photograph of the following items: A CASIO multifunction calculator. A ball point pen, uncapped. Various coins, apparently European, both copper and gold. Seems to be a theme illustration for a brochure or document cover treating finance, probably personal finance. version 4: …Various British coins; two of £ 1 value, three of 20 p value and one of 1 p value. … version 8: “A close-up photograph of the following items: A CASIO multi-function, solar powered scientific calculator. A blue ball point pen with a blue rubber grip and the tip extended. Six British coins; two of £ 1 value, three of 20 p value and one of 1 p value. Seems to be a theme illustration for a brochure or document cover treating finance - probably personal finance. "
New York: Old and New http: //newyorkoldandnew. blogspot. com/ An engagement experiment: “Search the NYPL digital library for an old photo of New York, and then find a photo of the same location, today”
1935
2007
1933
2007
1890
2007
1890
1900
1976
2008
The incomplete knowledge graph [Harsen Homestead, 72 nd Street, ca. 1888. ]
The incomplete knowledge graph Harsen Homestead ? ? ?
The incomplete knowledge graph Jacob Harsen, a farmer who started Harsenville, a village spanning 68 th Street to 81 st Street between Central Park West and the Hudson River. Source: http: //www. oneshermansquare. com/Historyof. Sherman. Square. aspx
The incomplete knowledge graph Crowd not only annotates historical archives but also contributes back to the knowledge graph [allowing easier annotation in the future]
New York Times, 1911
The incomplete knowledge graph Crowd participation not only annotates historical archives but also contributes back to the knowledge graph [allowing easier annotation in the future]
- Panos ipeirotis
- And other stories quality
- Contoh crowdsourcing
- Crowdsourcing
- Crowdsourcing journalism
- Prolific crowdsourcing
- Captcha
- Zahra literacy shed
- Valla de 8 paños
- Vallas de ocho paños
- Panos christakoglou
- The helios disaster
- Panos razis
- Panos christakoglou
- Panos charitos
- Geoffrey panos
- Panos navrozidis
- Partes del templo griego
- Contemporary management practices
- Perform quality assurance
- Basic concepts of quality assurance
- Types of repair
- Quality management in operations management
- What is tqm
- Short stories on change management
- Pmp quality vs grade
- What are quality standards in project management