Obtaining useful crowdsourcing results often requires more responses than can be easily collected. Reducing the number of responses required can be done by adapting to previous responses with "adaptive" sampling algorithms, but these algorithms present a fundamental challenge when paired with crowdsourcing. At the UW–Madison, we have built a powerful crowdsourcing data collection tool called NEXT (http://nextml.org) that can be used with arbitrary adaptive algorithms. Each week, our system is used by The New Yorker to run their Cartoon Caption contest (http://www.newyorker.com/cartoons/vote). In this talk, Scott Sievert will explain what NEXT is and it’s applications, architecture and experimentalist use.
Scott Sievert is a graduate student at the University of Wisconsin–Madison studying machine learning and optimization. He is an ambassador for the Center for Open Science and enjoys building and using a well designed software tool. His hobbies include skiing, backpacking and sailing.