Regression
Home Up Descriptive Correlation Regression Inferential t Test Analysis Chi Square Selection

Research Design in Occupational Education
Copyright 1997. James P. Key. Oklahoma State University
Except for those materials which are supplied by different departments of the University
(ex. IRB, Thesis Handbook) and references used by permission.


 

MODULE S3

REGRESSION

A prediction of the levels of one variable when another is held constant at several levels.
Based on average variation remaining constant over time due to the tendency in nature for extreme scores to move toward the mean.
Used to predict for individuals on the basis of information gained from a previous sample of similar individuals.

Regression Equation

= estimated y and is the value on the y axis across from the point on the regression line for the predictor x value. (Sometimes represented by or y.) This is the estimated value of the criterion variable given a value of the predictor variable.

a = the intercept point of the regression line and the y axis. It is calculated through the equation ; therefore, the means of both variables in the sample and the value of b must be known before a can be calculated.

b = the slope of the regression line and is calculated by this formula: If the Pearson Product Moment Correlation has been calculated, all the components of this equation are already known.

x = an arbitrarily chosen value of the predictor variable for which the corresponding value of the criterion variable is desired.

Example: A farmer wised to know how many bushels of corn would result from application of 20 pounds of nitrogen. The 20 pounds of nitrogen is the x or value of the predictor variable. The predicted bushels of corn would be y or the predicted value of the criterion variable.

Using the example we began in correlation:

Pounds of Nitrogen (x) Bushels of Corn (y)

x

y

10

-40

1600

30

-20

400

800

20

-30

900

40

-10

100

300

50

0

0

50

0

0

0

70

20

400

60

10

100

200

100

50

2500

70

20

400

1000

250

0

5400

250

0

1000

2300

We calculate the components of the regression equation beginning with b.

This gives us the slope of the regression line. For each 1.00 increment increase in x, we have a 0.43 increase in y. Next, we calculate a.

We can now plot our regression graph and predict graphically from it.

Or we can calculate the predicted values more accurately through the regression equation.

If we wish to know how much more corn to expect from a 35 pound application of nitrogen, we calculate:

Standard Error

The standard error for the estimate is calculated by the following formula:

The formula may look formidable, but we already have calculated all of the components except for squaring the .

 

This approximate value for the standard error of the estimate tells us the accuracy to expect from our prediction. Thus, for our prediction of 43.6 bushels from an application of 35 pounds of nitrogen, we can expect to predict a yield varying from 41 to 46.2 bushels with approximately 68% accuracy, to predict a yield varying from 38.4 to 48.8 with approximately 95% accuracy, or to predict a yield varying from 35.8 to 51.4 with 99% accuracy.

With the small numbers in this simple example and the large standard error of the estimate, you can see we have a wide range if our prediction is 99% accurate. This further points out the need for large samples and a high degree of relationship for accurate predicting. The size of the sample and the degree of the relationship determines the size of the standard error of the estimate to a great extent. Also, the accuracy of the predictions depend upon how well the assumptions are met.

Assumptions: (Same for correlation and regression)

1. Representative sample (Random)

2. Normal distribution for population

3. Interval measures

4. Linearity (Measures approximately a straight line)

5. Homoscedasticity (Equal variances)

Simple linear regression predicts the value of one variable from the value of one other variable. Multiple regression predicts the value of one variable from the values of two or more variables. Using two or more predictor variables usually lowers the standard error of the estimate and makes more accurate prediction possible. In our example if we could add soil type or fertility, rainfall, temperature, and other variables known to affect corn yield, we could greatly increase the accuracy of our prediction.

One caution. Due to the assumption of linearity, we must be careful about predicting beyond our data. If we predict beyond the information that we have known, we have no assurance that it remains linear or in a straight line. It might begin to curve and thus negate all our predictions in this region. Also, we must remember that the variables we are predicting must be like those upon which the regression equation was built or our prediction has no basis.

 

SELF ASSESSMENT

 

1. Define regression.

2. Definition Equation

=

a =

b =

3. The following are lists of competency scores of students on a vocational task alongside the number of hours they spent practicing and studying that task.

Student

Hours

Competency Rating

A

3

5

B

4

8

C

2

5

D

5

9

E

3

6

F

1

2

G

2

5

H

3

5

I

3

6

J

4

9

K

4

8

L

3

5

M

3

6

N

4

9

O

5

9

P

3

6

Q

3

5

R

2

5

S

2

5

T

1

2

 

Calculate a and b.
What is the predicted competence for a student spending 2.5 hours practicing and studying? 4.5 hours?
What is the standard error of the estimate?
Describe the accuracy of your prediction for 2.5 hours.

4. State the assumptions underlying linear regression.

5. Describe multiple linear regression.

6. State two precautions to observe when using linear regression.

Home Up Descriptive Correlation Regression Inferential t Test Analysis Chi Square Selection