Data Tools
Home Up Data Tools Data Tools 2 Reliability Historical Descriptive Experimental Qualitative

Research Design in Occupational Education
Copyright 1997. James P. Key. Oklahoma State University
Except for those materials which are supplied by different departments of the University
(ex. IRB, Thesis Handbook) and references used by permission.


 

MODULE R8

QUESTIONNAIRE AND INTERVIEW AS
DATA-GATHERING TOOLS

 

Questionnaire

A questionnaire is a means of eliciting the feelings, beliefs, experiences, perceptions, or attitudes of some sample of individuals. As a data collecting instrument, it could be structured or unstructured.

The questionnaire is most frequently a very concise, preplanned set of questions designed to yield specific information to meet a particular need for research information about a pertinent topic. The research information is attained from respondents normally from a related interest area. The dictionary definition gives a clearer definition: A questionnaire is a written or printed form used in gathering information on some subject or subjects consisting of a list of questions to be submitted to one or more persons.

Advantages

Economy - Expense and time involved in training interviewers and sending them to interview are reduced by using questionnaires.
Uniformity of questions - Each respondent receives the same set of questions phrased in exactly the same way. Questionnaires may, therefore, yield data more comparable than information obtained through an interview.
Standardization - If the questions are highly structured and the conditions under which they are answered are controlled, then the questionnaire could become standardized.

Disadvantages

Respondent’s motivation is difficult to assess, affecting the validity of response.
Unless a random sampling of returns is obtained, those returned completed may represent biased samples.

Factors affecting the percentage of returned questionnaires

Length of the questionnaire.
Reputation of the sponsoring agency.
Complexity of the questions asked.
Relative importance of the study as determined by the potential respondent.
Extent to which the respondent believes that his responses are important.
Quality and design of the questionnaire.
Time of year the questionnaires are sent out.

The questionnaire is said to be the most "used and abused" method of gathering information by the lazy man. because often it is poorly organized, vaguely worded, and excessively lengthy.

Two types of questionnaires

Closed or restricted form - calls for a "yes" or "no" answer, short response, or item checking; is fairly easy to interpret, tabulate, and summarize.
Open or unrestricted form - calls for free response from the respondent; allows for greater depth of response; is difficult to interpret, tabulate, and summarize.

Characteristics of a good questionnaire

Deals with a significant topic, a topic the respondent will recognize as important enough to justify spending his time in completing. The significance should be clearly stated on the questionnaire or in the accompanying letter.
Seeks only that information which cannot be obtained from other sources such as census data.
As short as possible, only long enough to get the essential data. Long questionnaires frequently find their way into wastebaskets.
Attractive in appearance, neatly arranged, and clearly duplicated or printed.
Directions are clear and complete, important terms are defined, each question deals with a single idea, all questions are worded as simply and clearly as possible, and the categories provide an opportunity for easy, accurate, and unambiguous responses.
Questions are objective, with no leading suggestions to the desired response.
Questions are presented in good psychological order, proceeding from general to more specific responses. This order helps the respondent to organize his own thinking, so that his answers are logical and objective. It may be wise to present questions that create a favorable attitude before proceeding to those that may be a bit delicate or intimate. If possible, annoying or embarrassing questions should be avoided.
Easy to tabulate and interpret. It is advisable to preconstruct a tabulation sheet, anticipating how the data will be tabulated and interpreted, before the final form of the question is decided upon. Working backward from a visualization of the final analysis of data is an important step in avoiding ambiguity in questionnaire form. If mechanical tabulating equipment is to be used, it is important to allow code numbers for all possible responses to permit easy transfer to machine-tabulation cards.

Guides for preparing and administering the questionnaire

Get all of the help you can in planning and constructing your questionnaire. Study other questionnaires and submit your own questionnaire to faculty members and class members for criticism.
Try your questionnaire out on a few friends or associates. This helps to locate unclear and vague terms.
Choose respondents carefully. It is important that questionnaires be sent only to those who possess the desired information - those who are likely to be sufficiently interested to respond conscientiously and objectively.
A preliminary card asking whether or not the individual would be willing to participate in the proposed study is recommended by some research authorities. This is not only a courteous approach but a practical way of discovering those who will cooperate in furnishing the desired information.
It has also been found that in many instances better response is obtained when the original request was sent to the administrative head of an organization rather than directly to the person who had the desired information. It is possible that when a superior officer turns over a questionnaire to a staff member to fill out there is some implied feeling of obligation.
If questionnaires are planned for use in public schools, it is imperative that approval of the project be secured from the principal or superintendent of the school.
If the desired information is delicate or intimate in nature, one must consider the possibility of providing anonymous responses. This will result in the most objective responses. If identity for classification purposes is necessary, the respondent must be convinced that the information will be held in strictest confidence.
Try to get the aid of sponsorship. Recipients are more likely to answer if a person, organization, or institution of prestige has endorsed the project.
Be sure to include a courteous, carefully constructed cover letter to explain the purpose of the study.
Some recipients are slow to return questionnaires. A courteous post card reminding an individual that the questionnaire has not been received will often bring in some additional responses.
An important point to remember is that questionnaires should be used only after all other sources on the topic to be researched have been thoroughly examined. The American Association of School Administration and the Research Division publish an annual bibliography, Questionnaire Studies Completed. This has been published since 1930.

Rules for proper construction of a questionnaire

Define or qualify terms that could easily be misunderstood or misinterpreted.
What is the value of the tools in your Vo-Ag shop? (Replacement, present, market, teaching value, etc.)
What are you doing now? (Filling out your stupid questionnaire.)
Be careful with descriptive adjectives and adverbs that have no agreed upon meaning, such as frequently, occasionally, and rarely (one person’s rarely may be another person’s frequently).
Beware of double negatives.
Are you opposed to not requiring students to take showers after gym classes?
Are you in favor of not offering Vo. Ag. IV in your Agriculture Program?

(One must study these questions carefully or answer improperly.)

Be careful of inadequate alternatives.

Married Yes ____ No ____

Employed Yes ____ No ____

(There are other answers that these types of questions do not answer, such as divorced, separated, union strikes, etc.)

Avoid double barreled questions.
Do you believe that students should be placed in separate groups for instructional purposes and assigned to special schools?
Should all Vo. Ag. I students take both wood and metal in shop? (One might be for one part of the question and opposed to the other.)
Underline a word to emphasize its importance.

The following illustration will serve to emphasize how a line under the word one wishes to emphasize can change a sentence from the standpoint of the respondent’s train of thought.

Were you there last night?

Were you there last night?

Were you there last night?

Were you there last night?

Were you there last night?

When asking for a rating, a point of reference is needed.
How would you rate campus dress today?
How would you rate student attitudes? (Compared to what?)
Avoid unfounded assumptions.
Are you satisfied with the salary you received last year? (A no answer might mean that I didn’t receive a raise last year or that I did get a raise, but I’m not satisfied.)
Phrase questions so that they are appropriate for all respondents.

What is your monthly salary?

What is your yearly salary?

(Both questions may not have a definite answer. Salaries could be ten months, eleven months, etc. Yearly salaries are subject to commission, seasonal changes, etc.)

Design questions that give complete possibilities for comprehension of responses.
Do you read the New York Times?
Do you watch the news on television?

(Neither question indicates much about the reading or viewing habits of an individual.)

Provide for a systematic quantification of responses.
What is your favorite TV program?
What is your favorite radio program?

(It is hard to summarize where the respondents pick a number of items from a list. A better way is to rank in order of preference. Example: Rank one through five the following radio programs. The items can then be tabulated by inverse weightings or points.)

Consider the possibility of classifying the responses yourself rather than having the respondent choose categories.
A student might miss his father’s occupation.

Unskilled labor ____

Skilled labor ____

Clearance work ____

Ask a child one or two short questions.

At what place does your father work?

What kind of work does he do?

(A younger child might not be able to choose the proper category in number one above. He could, however, answer the questions in the second example.)

The opinionnaire, or attitude scale, is an information form that attempts to measure the attitude or belief of an individual. How one feels or what he believes is his attitude, which can’t be measured or described. The researcher, therefore, must depend upon what the individual says are his beliefs and feelings. From the statement of his opinion his attitude is inferred.

 

Limitations of inferring attitude from expressed opinion

An individual may hide his real attitude and express socially acceptable opinions.
He may not really know how he feels about a social issue.
He may never have considered the idea seriously.
He may not know his attitude about a situation in the abstract and so may be unable to predict his reaction or behavior until confronted with a real situation.

 

Interview

An interview is a direct face-to-face attempt to obtain reliable and valid measures in the form of verbal responses from one or more respondents. It is a conversation in which the roles of the interviewer and the respondent change continually.

Advantages

Allows the interviewer to clarify questions.
Can be used with young children and illiterates.
Allows the informants to respond in any manner they see fit.
Allows the interviewers to observe verbal and non-verbal behavior of the respondents.
Means of obtaining personal information, attitudes, perceptions, and beliefs.
Reduces anxiety so that potentially threatening topics can be studied.

Disadvantages

Unstructured interviews often yield data too difficult to summarize or evaluate.
Training interviewers, sending them to meet and interview their informants, and evaluating their effectiveness all add to the cost of the study.

Structured interviews are rigidly standardized and formal.

The same questions are presented in the same manner and order to each subject.
The choice of alternative answers is restricted to a predetermined list.
The same introductory and concluding remarks are used.
They are more scientific in nature than unstructured interviews.
They introduce controls that permit the formulation of scientific generalizations.

Limitation of the structured interviews - Collecting quantified, comparable data from all subjects in a uniform manner introduces a rigidity into the investigative procedures that may prevent the investigator from probing in sufficient depth.

Unstructured interviews are flexible.

They have few restrictions.
If preplanned questions are asked, they are altered to suit the situation and subjects.
Subjects are encouraged to express their thoughts freely.
Only a few questions are asked to direct their answers.
In some instances, the information is obtained in such a casual manner that the respondents are not aware they are being interviewed.
Advantages of the unstructured interview:
One can penetrate behind initial answers.
One can follow up unexpected clues.
One can redirect the inquiry into more fruitful channels.
It is very helpful in the exploratory stage of research.
Disadvantages of the unstructured interview:
Difficult to quantify the accumulated qualitative data.
One usually cannot compare data from various interviews and derive generalizations that are universally applicable because of the nonuniform tactics employed.
Unstructured interviews are not ordinarily employed when testing and verifying hypotheses.

Factors to be considered before interviewing

Determine when to interview.
Determine if the respondent is telling the truth.
Consideration for sources of bias.

Four specific sources of error

Errors in asking questions occur whenever an inappropriate question is asked where the response to the question will not satisfy the objectives of the investigation.
Errors in probing occur when the interviewer does not allow the respondent sufficient time to respond or when he anticipates what the response will be.
Errors in motivating respondents can be a source of invalidity. Unless respondents are motivated by interviewers to answer questions to the best of their ability, they are likely to be uncooperative.
Errors in recording responses occur when the interviewer records the respondent’s answers inaccurately by omitting information.

 

Evaluation of a Questionnaire or Interview Script

Is the question necessary? How will it be used? What answers will it provide? How will it be tabulated, analyzed, and interpreted?
Are several questions needed instead of one?
Do the respondents have the information or experience necessary to answer the questions?
Is the question clear?
Is the question loaded in one direction? Biased? Emotionally toned?
Will the respondents answer the question honestly?
Will the respondents answer the question?
Is the question misleading because of unstated assumptions?
Is the best type of answer solicited?
Is the wording of the question likely to be objectionable to the respondents?
Is a direct or indirect question best?
If a checklist is used, are the possible answers mutually exclusive, or should they be?
If a checklist is used, are the possible answers "exhaustive"?
Is the answer to a question likely to be influenced by preceding questions?
Are the questions in psychological order?
Is the respondent required to make interpretations of quantities or does the respondent give data which investigator must interpret?

 

SELF ASSESSMENT

1. Construct a simple questionnaire of at least two questions each to illustrate an open or unrestricted form and a closed form.

2. Construct a simple questionnaire of at least two questions using the Likert scaling technique.

Home Up Data Tools Data Tools 2 Reliability Historical Descriptive Experimental Qualitative