Guidance for Survey Research

Before conducting a survey, there are a few key issues that you want to make sure to address. Asking yourself the following questions could save you a lot of time and energy and will also help guide you through the survey research process:

  • What is your research question?  Be sure you understand what information you want to obtain and how you intend to use the results.  How can your questions best be addressed/answered?  Would another form of data collection, such as a focus group or literature review, more efficiently answer your question(s).
  • Has anyone else already collected this information either through a survey or some other means?  You want to be sure not to duplicate any efforts that have already been made. Doing so may cause your intended respondents to ignore your request since they recently answered a similar one.
  • Are there other groups currently doing or planning to do similar research with whom you can work? 

SURVEY POPULATION

Once you have addressed the issues above, it is time to think about who you will be surveying. Referring back to your answers from question 1 above will help you in making this determination.  How specific is your question? Are you only interested in knowing what students in a particular course think? If so, and the enrollment in the course is small enough, you will most likely survey everyone in it. However, if the question you’re addressing is about a much larger group, for example all undergraduate students at the university, chances are you won’t be able to survey each and every student.  Instead, you will end up with a sample of the undergraduate students. Consider your survey instrument at this point.

If you’re doing a web or e-mail survey, you could attempt to get every student to participate. However, if you’re doing a paper survey, randomly selecting students to participate will save time, money, and paper from having to print, copy, and scan or enter data. If you are going to use a paper survey consider the time required for printing and copying and plan accordingly. At this point you should also be considering what your follow-up strategy will be. We generally recommend that you send a follow-up reminder email or letter about 2 weeks after the initial request. If you plan to do a second follow-up with non-respondents, allow at least 3 weeks to pass from the time of the initial request.

One of the most important things to consider when using a sample is size. Be sure that you have enough people in your survey to be able to generalize about the larger population that your sample comes from. If you’re concerned about getting a high enough response rate to produce valid results consider offering an incentive for completing the survey.  For more specific information on sampling, see one of the resources listed at the end of this guidance; Arlene Fink’s How to Conduct Surveys- A Step-by-Step Guide has a comprehensive yet easy to read section on this topic.

SURVEY CREATION

The creation of the survey is the next big step. Be sure to include either a cover letter or a concise description of your purpose at the beginning of the survey. Be clear about the response date and the time it will take for the respondents to complete the survey. Also take this opportunity to assure the respondents that their answers will be kept confidential. Keep the overall length of the survey as short as possible. Following these simple guidelines will help ensure a better response rate.

There are different types of question formats that can be used.  Some of the more common ones are Yes/No, choose one, choose many, Likert scale, and open-ended.  You can find some examples of these and more and guidance on determining which ones to use in Fink’s How to Conduct Surveys.  Other things to remember:

When using the Likert-scale question format be sure to use the same direction of the scale throughout the survey for similar questions (e.g., if 1 equals ‘strongly agree’ for one question, it should not equal ‘strongly disagree’ for another question).

  • Consider including ‘other’ or ‘N/A’ options where appropriate.
  • Ask any demographic questions at the end of the survey.
  • Be careful not to convey subjective importance of certain words or ideas by stressing them through bolding or underlining.
  • Be sure to test your survey on a few typical respondents before launching it. 
  • Particularly with paper surveys, make sure each survey is coded with a number to ensure respondents’ anonymity; so you can keep track of respondents; and to fix any data entry mistakes. 
  • Use extra caution to keep the identity of respondents confidential when dealing with private or sensitive information. Do not ask for unique identifiers such as name or social security number. If some sort of identification is necessary for tracking purposes, code each survey with a different number before it goes out.
  • If your research will involve using human subjects or requires identifiable private information, contact Bentley’s Institutional Review Board (IRB) before collecting information. If you have any question at all as to whether your research will require IRB approval, don’t hesitate to contact them or the Institutional Research & Planning office.   

 

DATA ANALYSIS AND REPORTING

An important beginning step for analysis of the results is to create a coding key. The best way to do this is to take a copy of the blank survey and assign numbers to each question and also to each possible response. For example, if a question required a respondent to check off how often they use the internet, either ‘frequently’, ‘occasionally’, or ‘never’ you would want to code the responses as 1, 2, or 3. These numbers will then be used for easier entry into whatever software tool you use for your analysis. Also, make sure you address how you will deal with questions that were left blank. We recommend that you do not code missing data with zeroes.  Doing so may incorrectly affect averages if you’re working with a scale variable.  Instead it is better to code these responses as system missing if using SPSS or simply omit that particular case from your results if doing your analysis in Excel.  If the variable is a categorical/nominal one, you may also assign these cases a number outside the realm of response options.  For example, if the question responses are coded as 1, 2, 3, or 4 consider coding blank responses as 99.

You should use a statistical software package such as SPSS or SAS.  However, if you’re looking to do just basic analyses such as computing averages, Excel is an easier tool to use and can serve the same purpose. In Excel using each column as a question and each row as an individual’s responses, once the data is entered you can do frequency tables, charts, and even standard deviations.
The last step of the survey research process is to write up a summary of the results, and when appropriate distribute it to the participants. Always include a description of the methodology used, the response rate, and any sampling error that you had to be aware of when interpreting the results.

For a more comprehensive overview of the steps in conducting a survey, use Alreck and Settle’s The Survey Research Handbook.  Arlene Fink does a good job of presenting the information in a concise and manageable format in her publication The Survey Handbook. The list below contains good resources on survey research:

  • The Survey Research Handbook, 3rd ed. – Pamela Alreck and Robert Settle, 2004.
  • How to Conduct Surveys, A Step-by-Step Guide, 3rd ed. –Arlene Fink, 2006.
  • The Survey Handbook, 2nd ed. – Arlene Fink, 2003.
  • How to Ask Survey Questions, 2nd ed. – Arlene Fink, 2003.
  • How to Manage, Analyze, and Interpret Survey Data, 2nd  ed. – Arlene Fink, 2003.
  • Constructing Effective Questionnaires – Robert Peterson, 2000.