ADVERTISEMENTS:
After reading this article you will learn about the problem and mathematics of sample size.
The Problem of Sample Size:
We shall now consider one of the trickiest problems relating to sampling, viz., the problem of sample size. “What should be the adequate size of the sample in relation to the size of population?” “How big ought to be a sample?” are questions often asked by research students. Xo decisive answer to this question can be given.
This is because the question of size can be answered only when we are sampling elements for the population in such a manner that each element has the same chance of being included in the sample, i.e., when we are adopting the probability design of sampling.
ADVERTISEMENTS:
Only the probability design makes possible the formulation of representative sampling plans. Hence, makes it possible of the formulation of representative sampling plans.
Hence, the question, “how large the sample should be in order to be representative of the population of a designated size?” presupposes the probability sampling procedure. Failing this procedure, representativeness of the sample howsoever large can only be a matter of hope and conjecture.
The general misconceptions in regard to the size of the sample is that the size of the universe from which the sample is drawn determines the number of cases needed to yield an adequate or representative sample of that universe.
We shall do well to note right away that the emphasis should be placed not upon the number of cases in the universe but on their number in the sample.
The Mathematics of Sample-size:
ADVERTISEMENTS:
The basic practical question “How to determine the sample-size which will yield the desired degree of precision as stipulated by the researcher for a given study?” The sampling problem is, of course, the same in all studies, i.e., to estimate or predict something about the population on the basis of knowledge of something about the sample.
The researcher must know what kind of statistics on the sample will serve the purpose, e.g., percentages, averages, standard deviation, etc., for such an estimation. This is important because different kinds of statistics are useful depending on the desired degrees of precision in sample returns which in turn are afforded by different sample-sizes.
Averages and percentages are the more commonly desired statistics, we shall therefore deal specifically with the question of sample-sizes corresponding to the desired degrees of precision in respect of averages and percentages.
Since the sample drawn by the researcher is only one of the many possible samples of the universe that he might have happened to choose, he needs to know how much reliance he can place on the sample as the representative of the ‘universe’ about which he wants to know something or with reference to which he wishes to generalize.
He needs to know how large the sample should be to give him a satisfactory level of precision. This calculation is possible by recourse to mathematics since in random sampling (probability sampling design) where every item in the universe has a specifiable probability of inclusion in the sample, the precision of prediction or estimate is related to the square root of the number of items in the sample.
Before proceeding with the calculation of the requisite size of the sample for a given study, it is necessary in practice, to secure some preliminary information about the population or universe.
If the researcher intends to use the sample to make an estimate of the average measure of particular characteristic in the universe, he needs to have some preliminary estimate to the standard deviation (dispersion) in the distribution of the values of items in the universe with respect to the given characteristic.
The researcher who comes to know the range of values (the spread) in respect of a particular characteristic in the universe can get a preliminary estimate of the standard deviation by dividing this range by 6, since the standard deviation of the (finite) universe may for all practical purposes be taken to be around 1/6 of the full range of variation.
In other words, the range of dispersion of a distribution may be taken to comprise 6 standard deviation units. The preliminary information about the universe may be had by means of a pilot study, results of past surveys, from reports published by statistical bureaus, reckoning of experts in the field, etc.
ADVERTISEMENTS:
The researcher, before proceeding to calculate the size of the sample, must decide the expected level of precision of the estimates. This expectation is based, in the main, on the purpose of the study.
In other words, the researcher must decide:
(a) How much error in the estimate to be derived from the sample (as compared to the true value, i.e., value of the ‘universe’) may be tolerated (called margin of error or limit of accuracy) and
(b) With how much assurance can it be said that the estimate will fall within this margin of error (called, level of confidence or probability).
It will be proper, however, to consider these in greater detail, presently:
(a) Margin of Error or Limit of Accuracy:
The basic question here is: ‘How much is the percentage or average to be secured from the study of the sample likely to vary from the true mean (of the population) and may still to be tolerated?’ The researcher might tolerate 5% error or he might require accuracy within a limit of 2%.
It all depends on how accurately or exactly he wants to know certain facts. Let us suppose that the researcher wishes to know in advance which of the two candidates contesting the election is going to win the seat. If the voting is going to be close, the researcher can afford to tolerate only a smaller error if he is to be practically certain.
He may, for example, set the permissible error at less than 2%. On the other hand, if the election appears to be one sided, and quite biased in favour of a particular candidate, the researcher may be able to predict the results even with a much larger error in the estimate.
If the sample-survey happened to reveal that 60% of votes would go in favour of a candidate, an error as high as 9% might be tolerated. In this case, even if the sample-poll had drawn the most unfortunate sample deviating 9% from the true value, the true value would still be 51%, i.e., 1% above the 50% which is the critical point.
Thus, both estimated value of 60% and true value of 51% would be above the critical point (i.e., 50%) and the prediction would be reliable.
(b) Probability or Confidence Level:
In addition to limit of accuracy, the researcher must also decide with reference to his study, how much confidence he would like to place in the sample-estimates being as close to the true estimate as to be within the limits of tolerance or accuracy set by him for the study.
In certain situations, he may want to be exceedingly sure that his estimates (based on the sample) will be within 51% of the true value whereas in certain other situations, he may be satisfied with a little lesser degree of assurance.
In social science research, two degrees of probability or confidence are very well-known and often used.
One of these is 0.95 level of probability, i.e., there will be 95 chances out of 100 that the sample estimate will not exceed the limits of tolerance or margin of error, and the second level is the 0.99 level, of probability, i.e., it is likely that in 99 chances out of 100 the sample estimate will not exceed the margin of error aimed at.
The level of confidence may even be set at 0.999, that is, the sample estimate would not deviate from the true value (of universe) beyond the limits of tolerance in 999 chances out of 1000. For certain purposes, the researcher may aim low and set the probability level at 0.67 (i.e., 2 out of 3).
The chances that a particular sample drawn for a study will yield an estimate of the universe which is within the margin of error, depend upon the variation among the samples that may be drawn from the universe. If the values secured from the samples tend to deviate considerably from the true value, then the chances of any given sample-value staying within the permissible limits of error are poor.
The standard error is the measure which tells us what the chances of a sample staying within the permissible limits are. It is a measure of variation in sampling estimate which could be expected in random sampling. Random samples tend to follow the laws of probability and the sample-estimates tend to cluster around the true value of the universe.
These estimates can be represented by a bell- shaped or normal curve. The mid-point of this curve represents the true value (of the universe) and the maximum variation or deviation of a random sample-estimate from this true value is about three times the standard error.
The standard error is thus about 1/6th of the entire range of random sampling variation. For all practical purposes, however, the standard error is taken as 1/4th of the range of variation, since the extreme variations occur very rarely.
Probability tables show that 95 out of 100 sample estimates can be expected to fall within the limit +2 and -2 standard errors. This means that if we have set our level of confidence or probability at 0.95, our problem will be to draw a random sample with a standard error which is about ½ (half) of our margin of error.
For a higher level of probability, we would have to draw a sample with a standard error, that is a still smaller fraction of the margin of error.
It should be noted that the standard error gets smaller (higher precision) as the samples get larger. To double the precision, the sample size must be multiplied by 4, i.e., increased four times; to treble it, the sample-size must be multiplied by 9; to quadruple it, by 16 and so on.
This only means that precision increases as the square-root of the number of cases in the sample. Statisticians have prepared tables which show the probability of sample estimates coming within the various standard error limits.
These limits are generally stated as + (plus) and – (minus). Such tables readily show, for instance, that 95% of the random sample estimates fall within the limit of +1.96 and -1.96 standard errors, about 68% of the estimates fall within the limits of+ 1 and -1 standard error and 99% of the estimates fall within the range of +2.57 and -2.57 standard errors, and so on.
In full consideration of (1) the margin of error and (2) the probability or confidence level, the researcher can proceed with the calculation of a desired sample-size. Mildred Parten has given the following formula for calculating the sample size, when the statistic to be estimated is the percentage. This is obviously a transposed variation of a standard error formula.
Size of sample = P.C.(100-P.C.)Z2 /T2
In the above formula, P.C. means the preliminary estimate of the percentage (from the universe).
Z means the number of standard error units which are found (from the normal probability table) to correspond to the required probability level.
T means the margin of error which may be tolerated (5% or 2%).
Parten has given the following formula for calculating the sample size for predicting or estimating the mean value of the universe in regard to a specified characteristic at a certain level of confidence and aimed at a given margin or error or limit of tolerance.
Sample size = (δ+Z/T)2
Where 8 stands for the preliminary estimate of standard deviation of the universe.
Z stands for the number of standard error units corresponding to the required probability or confidence level.
Let us take a concrete example and work out the sample-size. Suppose we wish to estimate the average annual income of families inhabiting a certain ‘middle class’ locality of a city.
Let us say, we have set our margin of error at Rs.100/-, i.e., we will tolerate the sample estimate within plus or minus 100 from the true mean of the population in respect of income. Suppose we have set the probability or confidence level at 0.95.
Suppose also that from a survey conducted a few years back, we estimate the standard deviation in respect of annual income of the population (locality) to be Rs.500/-. The value of Z, i.e., the standard error units corresponding to the probability of 0.95 is 1.96.
Substituting these values in the formula given above, we have
Size of simple =(500×1.96/100)2
= (9.8)2
= 95
This means that a random sample of 95 cases (families, which are the sample units) should give us an estimate of the mean of the given ‘universe’ within the set margin of error and at the desired level of confidence or probability, respectively, of Rs. 100/- and 0.95.
If we tighten the margin of error and set it at Rs. 50/-, the number of cases in the sample, i.e., the required size of the sample will be four times as large (i.e., 380) as the size required for the earlier margin of error (Rs. 100/-).
If another locality is characterized by greater homogeneity in respect of income and suppose, thus, that the standard deviation in income terms is only 100, the size of the sample for the above margin of error will be much lower.
In other words, the use of the formula illustrates the lesson namely, greater the homogeneity smaller the sample required and greater the accuracy aspired for, larger the sample-size needed.
The repeated use of such terms as the margin of error and level of confidence and other numerical expressions of probabilities and sample sizes, may tend to create the impression that a sample-size calculated by a formula will guarantee a desired precision.
It should be remembered, however, that the relations shown in the statistical tables of probability represent normal expectations in an ideal random sampling. But in as much as the actual sampling is rarely ideal, the relations expressed in tables cannot be expected to hold.
The general difficulty and rarity of ideal sampling should understandably make one skeptical about results which are exactly according to expectations.
This does not, however, mean that the researcher should not use or prefer the exact sample-size computed on the basis of the probability formula. In fact, this is precisely what he should do because it is his best bet. He should not, however, insist on this exact size if practical considerations make it inexpedient.
A substantially different approach to the problem of determining the desired sample-size is the ‘stability test.’ This consists in collecting data for relatively small sub- samples and keeping a running record of the distribution of the returns.
When after a point, the addition of more sub-sample does not change the results significantly, the researcher may assume that the total sample drawn thus far has become adequate, size wise. But this procedure may well be regarded as wasteful of time because it amounts in effect to a researcher engaging in a series of separate surveys spread over a considerable period of time.
It has been argued that this procedure is uneconomical in that more schedules are collected than are actually needed, since the tapering off to the point of approximate stability cannot be located with any certainty until the curve has maintained its level for a while.
But this does not seem to be a serious limitation when compared with the conservative practice of many reputable studies which collect more than the necessary/minimum number of items as a sample.
The main advantage of this type of stability test is that instead of depending upon calculations based on preliminary information, one simply increases the overall sample- size unit it is observed to be sufficient. The empirical check of watching the returns and stopping when they stabilize seem straightforward and convincing.
The chief danger of this procedure lies in the fact that the successive sub-samples collected are not likely to spread over the universe. Results may stabilize even though they do not represent the population.
In fact, the less representative the sub-sample, the more likely is the addition of more cases to yield the same result and throw up the appearance of stabilization. Unless sub-sample is a cross-section of the universe, there will not be a supersensitive sample upon which to observe the approaching stabilization.
The basic requirement of this procedure is that a growing representative sample must be available for observation. The expenses and the difficulty of collecting successive sub-samples which are spread over the universe are the major reasons why this is not likely to be representative.
The empirical stability test can be very effective, however, when the sub-samples are properly drawn and collected. The method is most appropriate for interview-surveys covering relatively small areas or community such as a town or a city because then, it is not so difficult or expensive to make each sub-sample a random sample of the population.
A more refined form of empirical control compared to stability test is a relatively recent development called Sequential Analysis. The general procedure involved, here, is to keep adding to the sample and at the same time keep testing the sample for significance until the minimum sample is accumulated that will provide the required level of significance.