QUESTION-BASED DATA COLLECTION METHODS
question based data-collection methods – general introduction
main types of question-based methods
some advantages of the face to face/personal interview
questionnaire construction – a rough guide
In these kinds of research, the researcher collects data by asking people who have experienced certain phenomena to reconstruct their experiences, describe their behaviour and express their preferences/attitudes. S/he may also ask about their knowledge of events.
Keeping in mind budget restrictions (time, money, personnel), the researcher must choose the data collection method that will elicit the most complete responses from a sample of individuals / organisations relevant to the research topic / objectives / questions.
The responses obtained are the data on which answers to the
research questions are based.
The responses obtained are the data on which answers to the research questions are based.
NB: distinguish between
your general research questions and objectives, which you will have spelt out in your research proposal, and
the specific questions you put to respondents in postal questionnaires and personal interviews.
Fowler (2002), Survey Research Methods 3rd edition ch. 5 – ‘Designing Questions to be Good Measures’;
Blaikie Designing Social Research pp. 232-5;
Questionnaire (self-administered), often postal.
Structured personal interview
Focused personal interview
‘Questionnaires’ are used in (non-face to face) postal or telephone surveys and in (face-to-face) structured personal interviews. The looser set of question headings used in the qualitative interviews are often called ‘interview schedules’.
‘Multi-method approach’ = using more than one method e.g. combination
of field research and a survey. Different kinds of questions i.e. questions that
produce both quantitative and qualitative forms of data,
may be used in the same interview or quetionnaire.
‘Multi-method approach’ = using more than one method e.g. combination of field research and a survey. Different kinds of questions i.e. questions that produce both quantitative and qualitative forms of data, may be used in the same interview or quetionnaire.
= an ‘impersonal’ method of data collection: whether this is desirable or otherwise depends on the kind of research strategy and research topic adopted
this involves a self-administered questionnaire in which access and return is through the post.
1. requires simple questions a postal questionnaire can be used to gather data only when the questions are straightforward enough to be understood and answered on the basis of printed instructions and definitions. In particular, the questions tend to be ‘closed’ and quantitative.
2. no opportunity for probing there is no opportunity to probe beyond the given answer, to clarify ambiguous answers, perhaps to appraise the non-verbal behaviour of respondents
3. no control over who fills out the questionnaire/ no control over the interview situation there is no way of knowing who has filled out the questionnaire: someone other than the intended respondent may have done so.
4. low response rate
it is often difficult to obtain a satisfactory response rate ( = the % in
the sample who return completed questionnaires). Typical response rates for
postal questionnaires without follow-up are 20% - 40%. The problem here is there
is no way of knowing if non-respondents are different from respondents in ways
that affect the findings.
FACTORS AFFECTING THE RESPONSE RATE / STRATEGIES TO INCREASE RESPONSE RATES
in the discussion below of personal interviews and postal questionnaires, it’s by and large assumed that the potential respondents are individuals to be interviewed or mailed in their own homes.
sponsorship providing information on who has commissioned the questionnaire often motivates respondents to complete and return them. (This is usually done in the ‘accompanying / covering letter’.) This information may convince respondents of
1. the legitimacy of the research
2. the usefulness of the research.
With officially sponsored research,
3. the perceived sanctions of failure to reply may also be a factor – these may be real legal sanctions, or they may be wrongly anticipated on the part of respondents. It is of course undesirable, and in general unethical, to suggest sanctions for failure to respond
appealing to respondents goodwill e.g. a student research project might mention the importance for their marks of a good response rate!
a reward e.g. a recent research project into drug use amongst teenagers in
The offer of entry into a draw overcomes this problem, but perhaps at the loss of any likely ‘payoff’ for participation.
Appeal to norms of common good / public spiritedness e.g. ‘making things better’
Pre-paid addressed envelope
It’s unreasonable to expect the respondent not only to spend time and effort to fill out the postal questionnaire but also to buy a stamp and write out the address on the envelope!
Timing of mailing / interviewing
Holiday periods are bad times to send out a postal questionnaire. The time of the day that a personal interview is arranged for is important, depending on who in the household it’s intended to question.
the most common strategy for postal questionnaires is to first send a reminder postcard to those who haven’t replied. Then perhaps a reminder, along with a substitute questionnaire.
However, sending a reminder may raise the question of the anonymity of the responses – as an alternative, the reminder could take the form of an announcement in the newspaper or other media.
Flexibility in the questioning process
on the research problem and research strategy, the interviews can vary from
highly structured [in the DEDUCTIVE RESEARCH STRATEGY] to highly unstructured
[ABDUCTIVE RESEARCH STRATEGY]. For focused and nondirective interviews, the
interviewer can clarify questions for the interviewee, probe for specific issues
raised in the course of the interview etc.
Control of the interview situation
The interview gives the
researcher much greater ‘control’ of the interview situation.
This isn’t as authoritarian as it sounds! -
Sequence of questions:
it may be desirable to ensure that questions are
asked in a particular sequence. If in a survey about facilities (childcare,
community centre, credit union etc.) in an area e.g. if two research questions
/ awareness of the existence of the facility (you want to find out how aware a
respondent is about the existence of facilities in their area, perhaps to gauge
the success or otherwise of a publicity campaign)
use of and attitude towards the facility
you need to ask them question 1 before presenting
them with a list of such facilities and asking them to comment on if or under
what circumstances they would use the facility
getting the right respondent
ensuring that the respondent him/herself has
answered the questions, rather than a relative – example of MS survey.
it may be important to fix a particular time when
the data was collected: this can allows more accurate interpretation of the
research results, especially when an event occurring around the time of the
interview could have influenced the respondent’s answers.
High response rate.
Response rates to personal interviews are usually higher than for postal questionnaires. Respondents who mightn’t bother filling out and returning a postal questionnaire will often cooperate with a request for a personal interview – there may be less effort involved on their part, e.g. if they have reading or writing difficulties.
central component of the data collection instrument in the postal questionnaire
or the structured personal interview, is the question.
answers to the questions asked = the data on which answers to the research
questions and objectives are built. The questions are the variables, which can
be correlated to look for associations between variables.
Different classifications possible
Objective/factual & subjective (‘objective’ = questions that generate
demographic economic data on gender, age, nos. in household, occupation etc.
& questions relating to possessions and behaviour e.g. car ownership, mode
of transport; ‘subjective’ = attitudes & opinions & beliefs e.g.
‘do you favour greater investment in public transport?’)
Behaviour & knowledge & beliefs & attitudes
STRUCTURE OF QUESTIONS
Closed-Ended & Open-Ended Questions
Contingency & Filter Questions
Closed ended: respondents are offered a set of answers
and asked to choose the one that most closely fits their situation/ views/etc.
The set of categories will depend on the research questions.
Quick & easy to answer
Easy to analyse
May introduce bias, by limiting the replies given or by offering
alternatives the respondent wouldn’t otherwise have thought of
Greater flexibility & a higher response rate
may be introduced by letting each individual option cover a range (of years,
Please indicate your age/ What age are you?
[ ] 25-35
[ ]64 or over
(spot the deliberate error!)
Don’t have overlapping possibilities in your response categories
Clearly indicate if respondents are allowed one or many responses
i.e. ‘one only’ or ‘all that apply’
Some questions may be irrelevant for a particular
respondent. E.g. if the respondent has no children, a question on age or level
of schooling of children doesn’t apply. The questionnaire may cover a number
of types of respondents, so rather than doing out as many separate
questionnaires as there are respondent types, you use one flexible,
filter questions are the signposts for detours
The contingency questions are the optional detours taken.
Asking/answering them is contingent on the response to the preceding filter
Reminder: any given questionnaire (= the data
collection instrument) may use a combination of question types (= mix of closed
and open ended questions).
This relates to the structuring of the response
categories of closed-ended questions.
The general format is to present all possible
answers and have respondents choose between the different categories.
What if the the respondents response doesn’t fit into any of the
answers supplied? = importance of the OTHER option + a ‘please specify’
For ease of data analysis, use ‘tick boxes’ or ‘circle
numbers’ rather than ‘tick blanks’. Sometimes blanks can be confusing,
both for the respondent and the data analyst.
Note that the response categories offered should
relate to the type of data being sought. Take the following example of a
question to obtain data about respondent age:
indicate your age/ What age are you?
[ ] 25-34
[ ]65 or over
If the research objectives
include description and exploration of the experiences of older people, then
’65 or over’ as above, may be too broad a category; instead you may want to
merge some of the other response categories (as being data irrelevant to the
[ ] 25-44
[ ]75 or over
A ranking (or ordinal) scale may be used to
structure responses to a question
Q. People found in possession of cannabis for
personal use should not receive jail sentences. Please indicate whether you:
Neither agree nor disagree
Other common response categories are:
two general patterns:
FUNNEL (going from the general question to the specific question).
Each successive question is related to the previous question and
has a progressively narrower scope.
allows interviewers to
avoid imposing a frame of reference before obtaining a response. E.g. if a
research objective was to explore whether there was a link between
respondents’ outlook on social & economic problems and the newspapers
they read, the sequence might be as follows:
Requires respondents to be motivated to respond (since you’re
starting off with ‘opinion’ questions
goes from the narrower question (eg aimed at establishing specific
facts) to the more general question (eg aimed at getting the respondents
overall judgement about these facts)
used where the survey topic isn’t of great interest to
respondents or where the researcher isn’t very knowledgeable about the
This sequence structure could apply to the entire
questionnaire, or to specific topics / sections within the questionnaire.
In general, it’s advisable to start off with
‘objective’ descriptive and neutral/non-threatening questions and then move
on to more ‘subjective’ and controversial questions.
The answers to the questions
asked = the data on which answers to the research questions and objectives are
A fundamental requirement of the question is that
it’s meaning should be clear to both the researcher and the respondent, at the
data analysis stage and when the findings are being interpreted. If the
questions - or any particular question - asked is unclear, then the validity
and reliability of the research is suspect.
Example of an unclear question: “Does
it seem possible or does it seem impossible to you that the Nazi extermination
of the Jews never happened?”
This was a question asked by the Roper polling
organisation in the
note the frequent use of a covering letter asking for access,
giving appropriate assurances of confidentiality and/or anonymity (many
methods textbooks will supply a specimen letter)
letting respondents see a draft of the report/interview
furnishing respondents / key informants with a final Report