OTHER DATA GATHERING TOOLS FOR A RESEARCH INVESTIGATION
Inventories are instruments that attempt to "take stock" of
one or more aspects of an individuals behavior rather than to measure in the usual
sense. Inventories have been used in educational research to obtain trait descriptions of
certain defined groups such as underachievers, dropouts, members of minority groups, etc.
They have also been used in research concerned with interrelationships between personality
traits and such variables as intelligence, achievement, and attitudes.
|The information obtained may be superficial or biased.|
|The difficulty of validating inventories limits their use as scientific instruments.|
To validate an inventory, one demonstrates that the scores obtained on
it agree highly with some other reliable device for measuring the factor under study.
A scale is a set of numerical values assigned to subjects, objects, or
behaviors for the purpose of quantifying the measuring qualities. Scales are used to
measure attitudes, values, and interests. They measure the degree to which an individual
possesses the characteristic of interest.
|Used as a measuring instrument.|
|Used to indicate a measuring instrument.|
|Used to indicate the systematized numerals of the measuring instrument.|
|Tests are scales, but scales are not tests.|
Attitude is an integral part of the personality (as to the way we
think, feel, perceive, and behave toward a cognitive objective). Attitude scales have
three major types which are the summated rating scales, equal appearing interval scales,
and cumulative or Guttman scales.
The summated rating scale is a set of attitude items, all of
which are considered of approximately equal attitude value and to which the subjects
respond with degrees of agreement or disagreement (intensity). The scores of the items of
such a scale are summed or summed and averaged to yield an individuals attitude
|U is the universe of items - means that there is no scale of items as such, but the
individuals responding to the items are scaled|
|Intensity of attitude expression - Respondents are asked to indicate whether they agree
very strongly (7), agree strongly (6), agree (5), disagree (3), disagree strongly (2),
disagree very strongly (1), no response (4)|
Equal-Appearing Interval Scales are used to scale the attitude
items. Each item is assigned a scale value, and the scale value indicates the strength of
attitude of an agreement response to the item. The universe of items is considered to be
an ordered set, items differ in scale value. The scaling procedure finds these scale
values. In addition, the items of the final scale to be used are so selected that the
intervals between them are equal, a most important and desirable psychometric feature.
Cumulative or Guttman Scale consists of a relatively small set of
homogeneous items that are unidimensional (or supposed to be). Unidimensional scale
measures one variable and only one variable. The scale gets its name from the cumulative
relation between items and the total scores of individuals.
Summated Rating Scales concentrate on the subject and their places on
the scale, most useful in behavioral research. Equal-Appearing Interval Scales concentrate
on the items and their place on the scale. Cumulative Scales concentrate on the
scalability of sets of items and on the scale position of individuals.
The value scale is a culturally weighted reference for a thing
or things for people, for institutions, or for some kind of behavior. Simply put, values
express the good, bad, shoulds, and oughts of human behavior.
The objective scale and items are independent and
non-independent. The types of objective scale include:
Those permitting one of two possible responses
Those permitting one of three or more possible responses
Those permitting more than one choice of three or more possible
|Rank Order Items|
|Forced Choice Items|
Direct observation is a measuring instrument used to measure such
traits as self-control, cooperativeness, truthfulness, and honesty. In many cases,
systematic direct observation of behavior is the most desirable measurement method. An
investigator identified the behavior of interest and devises a systematic procedure for
identifying, categorizing, and recording the behavior in either a natural or
Systematic Direct Observation
|Selecting the aspect of behavior to be observed. An observer cannot notice
everything that happens. Select first the behavior upon which the investigator wishes to
|Defining the behavior that fall within a category. Know in advance what will or will
not be classified as aggressive behavior, problem-solving behavior or any other
classification of interest.|
|Training observers. Observers must be trained for uniformity of interpretation and
standard application of the observation categories.|
|Quantifying observations. An observation system must include a standard method for
|Developing procedures to facilitate recording. Procedures to facilitate recording
must be developed to avoid errors of selectivity of memory. A useful technique is to
develop a coding plan that enables observers to record their observations with a single
letter or digit rather than in narrative form.|
The semantic differential (SD) is a method of observing and measuring
the psychological meaning of words, usually concepts. An actual SD consists of a number of
scales, each of which is a bipolar adjective pair. The bipolar adjectives are usually
seven-point rating scales. Each scale measures one or two of the basic dimensions or
factors that Osgood and his colleagues have found to be behind the scales: Evaluative,
Evaluative scales include pairs such as good-bad, bitter-sweet,
large-small, and dirty-clean. A second cluster has adjectives that seem to share strength
or potency ideas (strong-weak, rugged-delicate). A third scale is called activity
because its adjectives seem to express motion and action (fast-slow, hot-cold).
The first step in construction and use of SD is to choose the concepts
or other stimuli to be rated with bipolar adjectives. The researcher needs to choose a
number of concepts that are relevant to the research problem. A sample of concepts must be
judiciously chosen to represent some part of the semantic space. The second step is to
select appropriate scales or adjective pairs. Two main criterion determine the selection
including factor representatives and relevance to the concept used.
|It is a combination of the usual types of rating scales with factor analysis.|
|The technique is extremely flexible and simple to construct, administer, and score.|
|The semantic differential is subject to all of the limitation of rating scales, the
possibilities of faking responses, acquiescing (tendency to place marks in the middle
position), and having to mark a concept on a meaningless scale (is honesty more purple or
|Validity and reliability of semantic differential scales are generally satisfactory. The
validity studies show correlation coefficients of approximately .80 between the semantic
differential ratings and Thurstone, Likert, and Guttman scales. The test-retest
reliability of the semantic differential is reported to be about .90.|
|The semantic differential is a useful technique for measuring attitudes toward objects.|
Below describe how you feel about agriculture by placing a check
in one of the seven spaces between each word pair.
traditional _____ _____ _____ _____
_____ _____ progressive
simple _____ _____ _____ _____ _____ _____
like me _____ _____ _____ _____ _____
_____ unlike me
friendly _____ _____ _____ _____ _____
challenging _____ _____ _____ _____ _____
serious _____ _____ _____ _____ _____
stale _____ _____ _____ _____ _____ _____
work _____ _____ _____ _____ _____ _____
relaxed _____ _____ _____ _____ _____
clear _____ _____ _____ _____ _____ _____
unstructured _____ _____ _____ _____ _____
bright _____ _____ _____ _____ _____ _____
systematic _____ _____ _____ _____ _____
masculine _____ _____ _____ _____ _____
active _____ _____ _____ _____ _____ _____
accepting _____ _____ _____ _____ _____
closed _____ _____ _____ _____ _____ _____
Q methodology is used for the study of intrapersonal relations. The Q
technique is a set of procedures used to implement Q methodology. It is well suited to
small samples and makes use of the correlation between persons rather than the correlation
between tests. The Q technique centers particularly in the sorting of decks of cards
called Q sorts and in the correlations among the responses of different individuals to the
Steps in the application of Q methodology
|Begin with the development of items, usually 60 to 150, from theoretical formulation.|
|Each statement is printed on a separate card and presented to respondents who sort them
into piles. Generally, there are 11 piles, ranging from "most applicable to
oneself" to "least like oneself."|
|The Q technique requires each respondent to sort his deck of cards into a symmetrical
distribution with the number of cards to be sorted into each pile fixed by the
cards to be placed in each pile
|After sorting, each statement is assigned a score corresponding to the pile into which
it is placed.|
|If a number of individuals are involved in a study using just one sort, the correlation
between the individuals for each item or statement can be computed and a factor analysis
run to determine which individuals are responding similarly.|
|An alternative procedure is to have each person sort the deck numerous times under
different conditions, such as for idealized and actual self. The correlation between two
sorts can then be used as a measure of adjustment.|
|It is theoretically oriented.|
|Sortings can be analyzed quite objectively without entirely sacrificing the richness of
the usual clinical and much less objective methods.|
|Q can test the effects of independent variables on complex dependent variables.|
|It is useful in exploratory research.|
|It is interesting to subjects because the method is realistic as well as challenging.|
|It is not a method well suited to cross-sectional or large sample purposes.|
|Q placements violate the independence assumption in all forced-choice procedures.|
|Q is a forced-choice procedure which is unnatural because it requires the subject to
conform to an unreasonable requirement.|
|Important information on elevation and scatter may be lost with the forced Q procedure.|
The Conference technique is a face-to-face discussion of a topic of
|Experts are brought together at a common site.|
|The group brainstorms to generate as many ideas on the problem as possible. The only
rule regarding this step is that there are no negative reactions to any suggestions.|
|The experts then evaluate and rate the suggestions.|
|The most popular responses are determined, and an arbitrary number are chosen based on
natural breaks or logic.|
|Finally, the group discusses the strengths and weaknesses of the top suggestions and
ranks the final choices.|
One major drawback to this method of data gathering is the influence of
personalities as a strong factor in determining consensus.
The Delphi technique is used in the planning process, especially with
appraising the future political, economic, and social environment, ascertaining the role
of the organization in this environment, and anticipating and perceiving the needs and
requirements of client groups.
The Delphi technique is a means of securing expert convergent opinion
without bringing the experts together in face-to-face confrontation. This opinion of
experts is usually gained through the use of successive questionnaires and feedback with
each round of questions being designed to produce more carefully considered group
|A questionnaire is mailed to respondents who remain anonymous to one another. The first
questionnaire may call for a list of opinions involving experienced judgment, a list of
predictions, or a list of recommended activities.|
|On the second round, each expert receives a copy of the list and is asked to rate or
evaluate each item by some such criterion as importance, probability of success, etc.|
|The third questionnaire, which includes the list and ratings, indicates the consensus,
if any, and asks the experts either to revise their opinion or specify their reasons for
remaining outside the consensus.|
|The fourth questionnaire includes lists, ratings, consensus, and minority opinions. It
provides the final chance for revision of opinions.|
|It allows planners to get the views in a broad perspective rather than from an isolated
point of view.|
|Delphi in combination with other tools is a very potent device for teaching people to
think about the future of education in much more complex ways than they ordinarily would.|
|It is a useful instrument even for a general teaching strategy.|
|It is a planning tool which may aid in probing priorities held by members and
constituencies of an organization.|
|Delphi saves time and travel which are required to bring people together for a
|Delphi prevents personality biases from affecting the results.|
|Interpretation of the participants responses and the meaning of the importance of
the factors in planning is difficult.|
|It is unknown how the findings can be generalized to Delphis which cover a 30 year
extension into the future.|
|Delphi at present can render no rigorous distinction between reasonable judgment and
|It is difficult to determine the amount of bias injected into the results by the person
administering the Delphi.|
Nominal Group Technique
The committee chairman reiterates that the role of everyone present is
to contribute his perceptions, expertise, and experience to the identification of priority
problems. He emphasizes that the purpose is to identify and describe priority problems and
needs. He indicates that each member is to work individually without interacting verbally
with each other. The committee chairman distributes the problem or need identification
form to the members and ask them to respond in writing to the questions or statement on
the form, giving an example of the kind of response.
|Without discussion, silently and independently, each member lists on the form the
problems and/or needs for approximately 10 minutes. The chairman enforces silence by
requesting that those who have stopped writing not talk with others and think more deeply
for other possible items.|
|The recorder now asks each member to state an item from their list. The items are
recorded until all lists have been included. Discussion of items in not allowed and no
concern is given to overlap of items at this time. However, members are encouraged to
generate new ideas on their forms based on items presented by other group members.|
|The group now discusses for approximately 20 minutes the items listed for the purpose of
clarification, elaboration, combination of items, or addition of new items.|
|Without discussion and acting independently, each group member should select and
prioritize the items believed to be most critical. The recorder asks for each member to
give the items selected in priority order. The most critical items are determined by the
total amount of interest and votes. Discussion of voting and priority items continues
until a consensus is reached.|
Focus groups have been a mainstay in private sector marketing research
for the past three decades. More recently, public sector organizations are beginning to
discover the potential of this procedures. Educational and nonprofit organizations have
traditionally used face-to-face interviews and questionnaires to get information.
Unfortunately, these popular techniques are sometimes inadequate in meeting information
needs of decision makers. The focus group is unique from these other procedures; it allows
for group interaction and greater insight into why certain opinions are held. Focus groups
can improve the planning and design of new programs, provide means of evaluating existing
programs, and produce insights for developing marketing strategies.
|Involve people. It must be small enough for everyone to have opportunity to share
insights and yet large enough to provide diversity of perceptions. Focus groups are
typically composed of 6 to 10 people, but the size can range from as few as 4 to as many
|Conducted in series. Multiple groups with similar participants are needed to detect
patterns and trends across groups.|
|Possess certain characteristics. Participants are reasonably homogeneous and
unfamiliar with each other.|
|Provide data. Focus groups pay attention to the perceptions of the users and
consumers of solutions, products, and service. They are not intended to develop consensus,
to arrive at an agreeable plan, or to make decisions about which course of action to take.|
|Produce qualitative data.|
Grudens-Schuck, N., Lundy-Allen, B., & Larson, K. (2004, May).
Focus group fundamentals. Ames, IA: Iowa State University
Extension. Available at http://www.extension.iastate.edu/Publications/PM1969B.pdf
Larson, K., Grudens-Schuck, N., & Lundy-Allen, B. (2004, May).
Can you call it a focus group? Ames, IA: Iowa State University
Extension. Available at http://www.extension.iastate.edu/Publications/PM1969A.pdf
|It is a socially oriented research procedures.|
|The format allows the moderator to probe.|
|Discussions have high face validity.|
|Discussions can be relatively low cost.|
|The format can provide speedy results.|
|Focus groups enable the researcher to increase the sample size of qualitative studies.|
|The researcher has less control in the group interview as compared to the individual
|Data are more difficult to analyze.|
|The technique requires carefully trained interviewers.|
|Groups can vary considerably.|
|Groups are difficult to assemble|
|The discussion must be conducted in an environment conducive to conversation.|
Types of Focus Group Questions
|Opening Question. This is the round robin question that everyone answers at the
beginning of the focus group. It is designed to be answered rather quickly (within 10-20
seconds) and to identify characteristics that the participants have in common. Usually it
is preferably for these questions to be factual as opposed to attitude or opinion-based
|Introductory Questions. These questions introduce the general topic of discussion
and/or provide participants an opportunity to reflect on past experiences and their
connection with the overall topic. Usually these questions are not critical to the
analysis and are intended to foster conversation and interaction among the participants.|
|Transition Questions. These move the conversation into the key questions that drive
the study. The transition questions help the participants envision the topic in a broader
scope. They serve as the logical link between the introductory questions and the key
questions. The participants are becoming aware of how others view the topic.|
|Key Questions. These questions drive the study. Typically, there are two to five
questions in this category. These are usually the first questions to be developed and also
the ones that require the greatest attention in the subsequent analysis.|
|Ending Questions. These questions bring closure to the discussion, enable
participants to reflect back on previous comments, and are critical to analysis. These
questions can be of three types:|
|All Things Considered Question. This allows the participants to state their final
position on critical areas of concern.|
1. Define inventories.
2. List three advantages of inventories.
3. How do you validate an inventory?
4. Define scale.
5. List three characteristics of scales.
6. Name the three types of scales.
7. Name the three types of attitude scales.
8. Describe value scales.
9. List the types of objective scales.
10. Define direct observation methods.
11. List five steps for systematizing the direct observation methods.
12. Define semantic differential.
13. Name the three clusters of adjectives used in semantic
14. How do you construct and use the semantic differential?
15. List the characteristics of the semantic differential.
16. Define Q methodology.
17. How do you apply the Q methodology?
18. List three advantages of Q methodology.
19. List three disadvantages of Q methodology.
20. List the steps of a conference.
21. Define Delphi technique.
22. List the Delphi procedures.
23. List three advantages and three disadvantages of Delphi technique.
24. List the characteristics of a focus group.