Tag: Amazon’s Mechanical Turk

  • Amazon’s Mechanical Turk fights back

    Amazon’s Mechanical Turk fights back

    Innovation: Online army turns the tide on automation

    by Gareth Morgan

    Innovation is our regular column in which we highlight emerging technologies and predict where they may lead

    HumanHelpLiesWithin
    Human help lies within (Image: Noriyuki Araki/Flickr/Getty)

    Computer automation can take jobs away from people but, thanks to Amazon’s Mechanical Turk, humans are fighting back. AMT was inspired by the 18th-century inventor Wolfgang von Kempelen, who dazzled the Roman empress Maria Theresa with a chess-playing automaton. His secret: a human chess master hid inside the machine.

    In 2005, online retailer Amazon developed a version that uses a human workforce “hidden” on the internet to solve problems – for a modest price. Typically, the work undertaken is for organisations that need a little human smarts applied to bulk tasks, such as identifying objects in vast collections of images.

    An echo of von Kempelen’s Turk is found in the offices of robot maker Willow Garage, in Menlo Park, California. Some of the firm’s free-roaming robots rely on humans through AMT to help them get their bearings. Whenever one gets lost within the Willow Garage offices, it sends an image to AMT with a request for nearby objects to be identified, using the answers to establish its whereabouts.

    Get shorter

    At the User Interface Software and Technology symposium in New York City this week there are signs that AMT rivals computer automation on some tasks.

    Michael Bernstein at the Massachusetts Institute of Technology, and colleagues, have developed Soylent, an add-on for Microsoft Word that uses AMT workers to check language and grammar. In tests on text from Wikipedia entries, Word’s grammar checker picked up about a third of errors; Soylent spotted two-thirds.

    Solyent’s Shortn module tasks the online workers with shortening the text – to meet a word limit, for example. The Word add-on also boasts a macro-writing module, Human Macro, which lets a writer describe how they want to manipulate text – perhaps changing it into the past tense – without the complication of having to code their own set of instructions within Word.

    Say what you see

    Meanwhile, Jeffrey Bigham at the University of Rochester, New York, and colleagues, are using the image-analysis capabilities of AMT workers – predominantly based in the US and India – to help the visually impaired. They have created an iPhone app called VizWiz that gets AMT workers to interpret objects in the user’s environment – checking the small use-by date on a carton of milk, for example.

    The app is able to analyse the iPhone camera’s focal length and lens distortion, and data from the built-in accelerometer, to pick out a target object in sufficient detail before sending it. After identification, the result is read aloud.

    However, despite their lack of real brain power, there is one advantage that computers will continue to hold over their AMT rivals: computers don’t charge for their labour.

    References: Bernstein’s Soylent research paper (pdf); Bigham’s VizWiz research paper (pdf)

    , 05 October 2010

  • Amazon Mechanical Turk

    Amazon Mechanical Turk

    The Amazon Mechanical Turk (MTurk) is one of the suite of Amazon Web Services, a crowdsourcing marketplace that enables computer programs to co-ordinate the use of human intelligence to perform tasks which computers are unable to do. Requesters, the human beings that write these programs, are able to pose tasks known as HITs (Human Intelligence Tasks), such as choosing the best among several photographs of a storefront, writing product descriptions, or identifying performers on music CDs. Workers (called Providers in Mechanical Turk’s Terms of Service) can then browse among existing tasks and complete them for a monetary payment set by the Requester. To place HITs, the requesting programs use an open Application Programming Interface, or the somewhat limited Mturk Requester site.

    Requesters can ask that Workers fulfill Qualifications before engaging a task, and they can set up a test in order to verify the Qualification. They can also accept or reject the result sent by the Worker, which reflects on the Worker’s reputation. Currently, a Requester has to have a U.S. address, but Workers can be anywhere in the world. Payments for completing tasks can be redeemed on Amazon.com via gift certificate or be later transferred to a Worker’s U.S. bank account. Requesters, which are typically corporations, pay 10 percent of the price of successfully competed HITs (or more for extremely cheap HITs) to Amazon.[1]

    About the name

    The name Mechanical Turk comes from “The Turk”, a chess-playing automaton of the 18th century, which was made by Wolfgang von Kempelen. It toured Europe beating the likes of Napoleon Bonaparte and Benjamin Franklin. It was later revealed that this ‘machine’ was not an automaton at all but was in fact a chess master hidden in a special compartment controlling its operations. Likewise, the Mechanical Turk web service allows humans to help the machines of today to perform tasks they aren’t yet suited for.

    History of the service

    The service was initially invented for Amazon’s in-house use, to find duplicates among its web pages describing products.[1]

    The service was launched publicly on November 2, 2005, and is currently in beta. Following its launch, Mechanical Turk user base grew quickly, in part the result of the Slashdot effect. At that time there were a huge number of “Human Intelligence Tasks” (HITs) in the system. In early to mid November, 2005, there were tens of thousands of HITs, all of them uploaded to the system by Amazon itself for some of its internal tasks that required human intelligence. Web traffic grew to a massive amount near the beginning of December 2005. Since then, the number of HITs in the system has decreased, and by December 20, 2005 there were less than 100 groups of HITs on the average page load. By January, new types of HITs were set up, such as selecting the three best restaurants in a city, and third party HITs began to appear as well. As of April 2006, there were only the occasional batch of 25 HIT groups being offered, and the service had slowed to a crawl. As of January 2007 there were new HITS being offered of podcast transcribing and rating and image tagging (which is becoming very popular). In March 2007 there were reportedly more than 100,000 workers in over 100 countries.[1]

    In 2007, the service began to be used to search for prominent missing individuals. It was first suggested during the search for James Kim, but his body was found before any technical progress was made. That summer, computer scientist Jim Gray disappeared on his yacht and Amazon’s Werner Vogels, a personal friend, made arrangements for DigitalGlobe, which provides satellite data for Google Maps and Google Earth, to put recent photography of the Farallon Islands on the Mechanical Turk. A Slashdot effect sparked by Digg led to 12,000 searchers signing up, who were supplemented with imaging professionals working separately with the same data. The search was unsuccessful.[2] In September 2007 a similar arrangement was repeated in the search for aviator Steve Fossett. Satellite data was divided into 85 meter square sections, and Mechanical Turk users were asked to flag images with “foreign objects” that might be a crash site or other evidence that should be examined more closely.[1]

    Third party programming

    Programmers have developed various browser extensions and scripts designed to simplify the process of completing HITs. According to the Amazon Web Services Blog, however, Amazon appears to disapprove of the ones that automate the process 100% and take out the human element. Accounts using so-called automated bots have been banned.

    Related systems

    Main article: Crowdsourcing

    MTurk is comparable in some respects to the now discontinued Google Answers service. However, the mechanical Turk is a more general marketplace that can potentially help distribute any kind of work tasks all over the world. The Collaborative Human Interpreter by Philipp Lenssen also suggested using distributed human intelligence to help computer programs perform tasks that computers cannot do well. MTurk could be used as the execution engine for the CHI.

    Criticism

    Because HITs are typically simple, repetitive tasks and users are paid often only a few cents to complete them, some have criticized Mechanical Turk as a “virtual sweatshop.” Workers have no recourse if companies refuse to pay them for good work. Requesters do not have to file tax forms, and avoid minimum wage, overtime, and workers compensation laws. Workers, though, must report their income as highly-taxed self-employment income. However, at least some workers on Mechanical Turk are people who are middle class and do the work to end boredom or for fun.[3]

    References

    ^ Steve Silberman. “Inside the High-Tech Search for a Silicon Valley Legend”, Wired magazine, July 24 2007. Retrieved on 2007-09-16.

    ^ I make $1.45 a week and I love it Salon.com. July 24, 2006.

    External links

    Official website

    Wired Magazine story about ‘Crowdsourcing’ June, 2006

    Business Week Article on Mechanical Turk

    New York Times Article on Mechanical Turk