Description:
Occasionally we receive inquiries about an odd-looking power curve that occurs when calculating power for proportions tests when the sample sizes are small. This power curve is sometimes called the saw-tooth power function because of its jagged appearance.
Suppose we intend to test whether the proportion of proper diagnoses of a disease is greater than 0.6. We want to investigate the power of the test for various sample sizes if we assume that the true proportion of proper diagnoses is 0.8. We decide to look at the power for a range of sample sizes between 30 and 60.
When we examine the chart of power plotted against sample size, there is an obvious up-and-down, jagged trend in the power, although the overall power trend is increasing with increasing sample size.
The jagged trend occurs because of the limited rejection region that is available for each sample size. Because only integer values can define the rejection region, the target alpha can rarely be achieved exactly from the available discrete rejection regions. Thus, a cutoff resulting in an alpha less than the target must be used instead.
For example, if we look at the rejection region for the sample size of 30, the nearest rejection region resulting in an alpha less than or equal to 0.05 is to reject if the number of proper diagnoses is 23 or greater. In this case, the actual alpha is 0.0435 and the power is 0.76. For a sample size of 31, however, the actual alpha is only 0.0330, and the resulting power is 0.73. In this case, the power decreased, even though the sample size increased. This is due to the limited rejection region that is available for a sample size of 31. Going to a sample size of 32, the actual alpha is lower still, and the power is also lower. When we examine a sample size of 33, the cutoff is 25, just as it was for 32, but the resulting alpha jumps back up to 0.0444, and there is a very large increase in power – from about 0.7 to 0.8.
As we go down the various sample sizes, this pattern continues, with decreases in alpha and power, and then an abrupt jump in both. This pattern is reflected in the jagged power values that are shown in the power curve.
In summary, because rejection regions are limited to integer values, there is a fluctuation in alpha levels across sample sizes, which, in turn, is reflected in a fluctuation in power values.
This phenomenon occurs in most of the single proportion tests, and to a lesser degree in the case of two proportions tests.
