Getting Started with Surveys and Market Research
--
With Amazon Mechanical Turk (MTurk), you can recruit participants from MTurk’s large and diverse base of Workers for your survey or online research study. Here is a brief summary of the most common steps involved in getting a Survey task live on MTurk, followed by more detail on each:
- Select your Survey Template: Qualtrics or SurveyMonkey users will probably want to check out the Survey Link template. There’s a tutorial here, or you start using it directly here. You can learn more here.
- Set the # of respondents and reward: tasks on MTurk are called HITs, and responses from Workers are called Assignments. A typical survey consists of one HIT with hundreds or thousands of Assignments. You set the reward a Worker will earn for responding. You can learn more here.
- Setting Worker criteria (optional): you can refine the cohort of eligible Workers by dozens of attributes including age, gender, education level, marital status, and more. You can learn more here.
- Linking MTurk with your Survey: if you’re using Qualtrics or Survey Monkey, you’ll provide MTurk with the link (URL) to your Survey. This is what lets Workers find it. This tutorial steps you through how to do this. You can learn more here.
- Publishing your Survey to Workers: here you’ll actually publish your survey so that Workers can discover it, accept it, respond to it, and submit the HIT for you to review, approve, and pay. You can learn more here.
Step 1: Select your Survey Template
If you are using a survey builder tool like Qualtrics or SurveyMonkey, you’ll probably want to check out the MTurk Survey Link template. This template lets you direct Workers to your survey via a web link (URL). You can learn more about the Survey Link template here, or start using it directly here.
Alternatively, if you aren’t using one of these tools and would prefer to host your survey with MTurk, check out the Survey template. You can access it directly here.
Step 2: Setting the number of respondents and the reward
MTurk makes it easy to pay respondents for completing your survey. Your survey is considered a single task (in MTurk, these are called “Human Intelligence Tasks” or “HITs”). Each response to your survey is called an Assignment. For example, if you have a single survey and need 500 respondents, you’d specify “500 assignments per HIT” as shown below.
In the above example, each Worker is given a maximum of 1 hour to complete the survey (otherwise the survey will be withdrawn and given to another Worker) so it’s important to ensure you give the Worker adequate time to complete the survey. Likewise, in the above example, MTurk will expire your HIT and stop collecting more responses after 7 days or as soon as 500 respondents are received — whichever occurs first.
You’ll have an opportunity to review each response by Workers before approving and paying them. You can specify the length of time you’ll need to review and approve, up to 30 days maximum. In the above example, you’ll have 3 days to review responses before MTurk will automatically approve them on your behalf and issue the task payment to Workers.
Step 3: Finding the right Workers
You can fine-tune the criteria of your respondents by adding Qualifications to your HITs. A Qualification is an attribute that a Worker possesses to indicate a particular skill, characteristic, or reputation
To use this, click (+) Add another criterion:
You can then select from dozens of potential demographic or behavioral attributes to specify your respondent’s criteria. We call these Premium Qualifications. You can learn more about Premium Qualifications here.
Step 4: Linking and Executing your Survey
If you’re hosting your survey with a tool like Qualtrics, SurveyMonkey, Google Forms, or equivalent, you simply need to include the link to your survey so that Workers know where to find it. To do this, you’ll edit the layout of your HIT.
To edit this, simply replace the link above with your link and remove the “Template note for Requesters” section. It should look something like this when you’re done:
You’ll notice a “survey code” in the template. Because Workers are discovering and accepting the task (the “HIT”) through MTurk, but are visiting another site (like Qualtrics, SurveyMonkey, or similar) we need a way for the Worker to come back to MTurk and confirm they’ve completed the survey.
A common approach is for the Requester to provide a code at the end of the survey. Workers accept the HIT, visit the survey link, complete the survey, get the code, come back to MTurk, and provide the code they received. This is your confirmation that the Worker did, in fact, complete the survey. There are lots of ways to generate this code. Check out this tutorial for one approach using Qualtrics.
Step 5: Publish your Survey to Workers
Now that you’ve prepared everything for your Survey, the only remaining steps involve publishing it so that Workers can discover it, accept it, respond to your questions, and submit the HIT for payment.
To start, simply view your list of Projects (your newly-created Survey should be first on the list). You can view your list of Projects here.
Click the “Publish Batch” button. You’ll be shown a preview of your HIT. This is what Workers will see when they discover your survey on MTurk.
If everything looks ok, simply click next to view a summary of the survey, the HITs, and the cost. On this screen, you can add or select the credit card you wish to use to pay for the results, and click Purchase & Publish to finish.
That’s it. Congratulations! Now your survey is live, and you should begin seeing Workers submitting their responses shortly.
