Linear Regression and the Correlation Coefficient

The high-low method gives a quick answer, but it throws away every observation except two. Linear regression analysis does better. It uses every pair of data to draw the line of best fit, and a companion measure, the correlation coefficient, tells you how much trust to place in that line. Together they are the standard way to estimate fixed and variable cost, and to project a trend into the future.

This topic is a regular source of long calculation questions worth 10 to 15 marks, and of short questions that ask you to explain what a correlation coefficient means. This page gives the formulae, a repeatable layout for the working, three fully worked examples, and the interpretation rules you need to score full marks.

Purpose of linear regression analysis

Linear regression analysis is a statistical technique for calculating a line of best fit from data in pairs: for each value of x there is an associated value of y. It has two main uses.

• Cost estimation. From historical total cost data, it estimates fixed cost (a) and variable cost per unit (b). It is an alternative to the high-low method.
• Forecasting. It can project a historical trend, such as sales, into the future, on the assumption that the trend rises at a steady rate in a straight line.

Regression compared with high-low analysis

In short, regression is the better technique because it is more reliable and its reliability can be measured.

The regression formulae

Given n pairs of data, the line of best fit y = a + bx is found from:

Setting out the working

1. List the pairs in two columns: x (activity) and y (cost).
2. Add a column for x² and fill it in by squaring each x.
3. Add a column for xy and multiply each x by its y.
4. Add the total of every column.
5. Substitute the totals into the formula for b, then into the formula for a.
6. Write the equation y = a + bx and use it for estimates.

Be careful with the denominator: it is n∑x² minus the square of the total of x, which is (∑x)². It is not ∑x² minus anything else, and it is not the total of x squared item by item.

Worked example: cost estimation by regression

A firm records output (x, in thousand units) and total cost (y, in ₦000) for five periods. The table includes the extra y² column that is needed later for the correlation coefficient.

1. b.
   (5 × 2,785 − 36 × 366) ÷ (5 × 282 − 36²) = (13,925 − 13,176) ÷ (1,410 − 1,296) = 749 ÷ 114 = 6.570.
2. a.
   366 ÷ 5 − 6.570 × (36 ÷ 5) = 73.2 − 47.31 = 25.89.
3. Equation:
   y = 25.89 + 6.570x.
   Because x is in thousands of units and y in ₦000, fixed cost is about ₦25,890 a period and variable cost is about ₦6.57 per unit.
4. Estimate at x = 8 (8,000 units):
   25.89 + 6.570 × 8 = 25.89 + 52.56 = 78.45, about ₦78,450.
5. Correlation:
   r = 749 ÷ √[114 × (5 × 27,778 − 366²)] = 749 ÷ √(114 × 4,934) = 749 ÷ √562,476 = 749 ÷ 750 = +0.999.
   This is a very strong positive correlation.

For comparison, the high-low method on the same data would use only periods 1 and 5: variable cost = (91 − 52) ÷ (10 − 4) = 6.5 and fixed cost = 52 − 6.5 × 4 = 26. The answers are close here because the data fits a straight line very well. With scattered data the two methods can differ more, and regression is the more trustworthy.

Forecasting other variables with a time trend

Regression can forecast things other than cost, such as sales or demand. Let x be the time period (1, 2, 3 and so on) and y the variable. The calculation is the same, and the equation estimates y for future periods. It rests on the assumption that the straight-line trend of the past will continue.

Example. Sales (₦m) over five years: year 1, 20; year 2, 24; year 3, 27; year 4, 31; year 5, 36. n = 5; ∑x = 15; ∑y = 138; ∑xy = 20 + 48 + 81 + 124 + 180 = 453; ∑x² = 55. b = (5 × 453 − 15 × 138) ÷ (5 × 55 − 15²) = (2,265 − 2,070) ÷ (275 − 225) = 195 ÷ 50 = 3.9. a = 138 ÷ 5 − 3.9 × (15 ÷ 5) = 27.6 − 11.7 = 15.9. So y = 15.9 + 3.9x, and sales in year 6 are forecast at 15.9 + 3.9 × 6 = ₦39.3m.

Correlation

Regression can fit a line to any pairs of data. That does not prove that a real relationship exists, and the line says nothing about how well it fits. Correlation measures how close the points on a scatter graph are to the line of best fit. If all the points are very near the line, a relationship between x and y is strongly suggested.

But correlation is not causation: two things may move together without one causing the other.

Degrees of correlation

• Positive correlation: y rises as x rises. Perfect positive: all points lie exactly on a straight upward line, so x and y rise in direct proportion.
• Negative correlation: y falls as x rises. Perfect negative: all points lie exactly on a straight downward line.
• Uncorrelated: no relationship can be seen between the variables.

The correlation coefficient, r

The formula for r is normally given in the examination:

It looks daunting, but it is similar to the formula for b. The top line is identical. The only new quantity is [n∑y² − (∑y)²], which needs one more column for y².

r must always lie between −1 and +1. If you calculate a value outside this range, you have made an error. For cost estimation, an r close to +1 suggests the estimates are very reliable.

Using the five-period data in the worked example above, the extra column of y² gives ∑y² = 27,778. The top line of the formula is 5(2,785) − (36)(366) = 749, the same as in the formula for b. The two terms under the square root are 5(282) − 36² = 114 and 5(27,778) − 366² = 138,890 − 133,956 = 4,934. So r = 749 ÷ √(114 × 4,934) = 749 ÷ 750 = +0.999. A value this close to +1 shows that cost rises almost exactly in step with output, so the cost equation can be used with confidence. A value above +0.90 is the usual benchmark for good correlation.

The ideas in more depth

Why two decimal places are not always enough

Rounding b too early can move a noticeably, and it can move forecasts far from the data even more. In the worked example, b is exactly 749 ÷ 114 = 6.5702. If you round it to 6.6, then a = 73.2 − 6.6 × 7.2 = 25.68 instead of 25.89, and a forecast for x = 50 changes from 25.89 + 6.5702 × 50 = 354.4 to 25.68 + 6.6 × 50 = 355.7. Keep at least three decimal places for b in your working and round only the final answer.

Checking the arithmetic

Most marks lost in regression questions come from slips in the table. Check each column total by adding it twice. Check that b is positive if cost rises with activity. And check that your line passes close to the data: the line must pass through the point (average x, average y). Here that is (7.2, 73.2): 25.89 + 6.570 × 7.2 = 73.2. ✓

A note on the printed solutions

If you practise with other textbooks or past papers, compare your totals with theirs before you compare the final answer. A single wrong total, such as using a total of xy from the wrong row, changes b, a and r together.

Interpreting r in an exam answer

A complete answer states four things: the value; its direction (positive or negative); its strength (close to 1 means strong); and what it means for the decision. Example: “r = +0.97 shows strong positive correlation between output and cost. The points lie close to the regression line, so the equation is reliable for estimating costs within the range of the data.

However, correlation does not prove that output causes the cost, and estimates far outside the observed range may be less reliable.”

Limits of regression

• It assumes a straight-line (linear) relationship.
• Forecasts outside the range of the data (extrapolation) are less reliable.
• A high r on a few data points may be a coincidence.
• Past trends may not continue.

Worked example: full calculation with interpretation

1. Table.
   ∑x = 2 + 4 + 5 + 7 + 8 = 26. ∑y = 30 + 41 + 45 + 59 + 64 = 239. ∑xy = 60 + 164 + 225 + 413 + 512 = 1,374. ∑x² = 4 + 16 + 25 + 49 + 64 = 158. ∑y² = 900 + 1,681 + 2,025 + 3,481 + 4,096 = 12,183. n = 5.
2. b.
   (5 × 1,374 − 26 × 239) ÷ (5 × 158 − 26²) = (6,870 − 6,214) ÷ (790 − 676) = 656 ÷ 114 = 5.754.
3. a.
   239 ÷ 5 − 5.754 × (26 ÷ 5) = 47.8 − 29.92 = 17.88.
4. Equation:
   y = 17.88 + 5.754x.
   Fixed cost about ₦17.88m a period, and variable cost about ₦5.754m per thousand units.
5. Estimate for 6,000 units (x = 6):
   17.88 + 5.754 × 6 = 17.88 + 34.52 = ₦52.40m.
6. r.
   656 ÷ √[114 × (5 × 12,183 − 239²)] = 656 ÷ √(114 × 3,794) = 656 ÷ √432,516 = 656 ÷ 657.7 = +0.997.
7. Interpretation. r is very close to +1, so there is strong positive correlation. The line fits the data very closely and can be used for estimating cost with confidence within the observed range of 2,000 to 8,000 units.

How well does the line fit? The coefficient of determination

The correlation coefficient r measures the strength of the linear relationship. Squaring it gives the coefficient of determination (r²), which has a more direct meaning: the proportion of the variation in y that is explained by the variation in x. In the earlier worked example, r = 0.999, so r² = 0.998: 99.8% of the variation in cost is explained by activity, leaving only 0.2% to other causes.

If a different data set gave r = 0.80, r² would be 0.64, meaning that 64% of the variation is explained by the line and 36% is not. A cost equation with r² of 64% is usable for rough estimates but would need caution; one with 99% is highly dependable.

A high r or r² shows that x and y move together, but it does not prove that x causes y. Cost may rise with activity because activity drives it, or both may be rising together because of inflation. The analyst must judge whether a causal link is sensible.

Regression and high-low compared on one data set

Use five months of data: output (units) 10, 20, 30, 40 and 50, with total cost (₦000) 400, 410, 470, 480 and 560.

• Regression:
  b = (5 × 73,500 − 150 × 2,320) ÷ (5 × 5,500 − 150²) = (367,500 − 348,000) ÷ 5,000 = 3.9. a = 2,320 ÷ 5 − 3.9 × 30 = 464 − 117 = 347.
  Equation: y = 347 + 3.9x.
• High-low:
  variable cost = (560 − 400) ÷ (50 − 10) = 4.0.
  Fixed cost = 400 − 4.0 × 10 = 360.
  Equation: y = 360 + 4.0x.
• Correlation:
  r = 19,500 ÷ √[5,000 × (5 × 1,093,000 − 2,320²)] = 19,500 ÷ √(5,000 × 82,600) = 19,500 ÷ 20,323 = +0.960, so r² = 0.921.

Estimate cost at 60 units: regression 347 + 3.9 × 60 = ₦581,000; high-low 360 + 4.0 × 60 = ₦600,000. The estimates differ by ₦19,000 because high-low used only the two extreme months, whose figures (400 and 560) happen to lie slightly off the general pattern. Regression uses all five points and is more reliable, which is why it is preferred when the data is scattered.

Interpolation and extrapolation

Using the line to estimate within the range of the data is interpolation (for example, 35 units: 347 + 3.9 × 35 = ₦483,500). Estimating outside the range is extrapolation (60 units, above the highest observed 50). Interpolation is generally safe. Extrapolation is riskier because cost behaviour may change outside the observed range, for example when a new shift raises fixed costs or bulk discounts lower variable cost.

The further the estimate lies from the data, the less reliable it is. Examiners often ask for a comment on this.

Regression in a spreadsheet

A spreadsheet does the arithmetic in one step. If x values are in cells A2:A6 and y values in B2:B6, then =SLOPE(B2:B6,A2:A6) gives b, =INTERCEPT(B2:B6,A2:A6) gives a, =CORREL(B2:B6,A2:A6) gives r, and =RSQ(B2:B6,A2:A6) gives r².

In the examination you must do the working by hand, but knowing these functions lets you check your answer and shows how the technique is used in practice. A scatter chart with a trendline displays the fitted line and can show the equation and r² on the chart.

Step-by-step routine for a regression question

1. Decide which variable is x (the cause, such as activity) and which is y (the effect, such as cost). Mixing them up gives wrong answers.
2. Draw up columns for x, y, x² and xy (and y² if r is needed). Count the observations, n.
3. Total each column. Check each total twice.
4. Calculate b first, then a.
5. Write the equation and state the units, such as “fixed cost ₦347,000 and variable cost ₦3,900 per unit”.
6. Use the equation to estimate, noting the assumptions.
7. If asked, calculate r and comment on it.

Limitations of regression analysis

• It assumes a linear relationship. If cost behaves in steps or curves, the straight line is only an approximation.
• It relies on past data, which may not represent future conditions (price changes, new technology).
• A few unusual observations (outliers) can distort the line. Examine the data for one-off events before fitting.
• With very few observations, the line is unreliable.
• It shows association, not proof of cause.
• Using more than one explanatory variable (multiple regression) needs more advanced methods than are required here.

Common mistakes in the exam

• Calculating a before b. You need b first.
• Squaring each item in the denominator wrongly: it is n∑x² − (∑x)², where (∑x)² is the square of the total.
• Mixing up ∑x² (sum of the squares) and (∑x)² (square of the sum).
• Arithmetic slips in the xy and x² columns. Check each total by adding the column twice.
• Forgetting the units when x is in thousands, so that the answer is wrong by a factor of 1,000.
• Giving r outside the range −1 to +1.
• Writing that a high r proves that x causes y.
• Interpreting r without saying whether it is positive or negative, or how strong it is.

Quick revision

• b = (n∑xy − ∑x∑y) ÷ (n∑x² − (∑x)²). a = ∑y/n − b∑x/n. Find b first.
• Columns: x, y, x², xy (plus y² for r).
• Regression uses all data and its reliability can be measured. High-low uses two points.
• r = (n∑xy − ∑x∑y) ÷ √[(n∑x² − (∑x)²)(n∑y² − (∑y)²)]. Range −1 to +1.
• +0.90 to +1: good correlation. 0: none. Correlation is not causation.
• Time-trend forecasting: let x be the period number.

Practice questions

These are original questions written in the style of the Institute of Chartered Accountants of Nigeria (ICAN) exam. Try each one before you open the answer.

Section A style: multiple choice

Section B style: written questions

A further written question

Model answers

Spotted a mistake? Tell us and we will check it.