Apart from random sampling error, there are two other sources of error in making inferences about a population from a sample. The first involves response bias and the second concerns the representativeness of the sample.

Response Bias

Almost all questions about people's opinions, attitudes, expectations, or preferences involve response biases of various kinds. When 44% of the voters sampled in a public-opinion poll in June say they will vote for the Republican presidential candidate, and when the actual vote in November turns out to be 53% for the Republican, the polls are often said to have been 11 wrong." However, the discrepancy between 44% and 53% can seldom be accounted for by sampling error. Instead, it is mostly due to the fact that respondents did not do in November what they said they would do in June.

Such response bias stems from a number of factors. First, opinions change over time. Second, people do not always say what they will do or have done. Third, people cannot answer certain questions realistically; for example, they seldom know how much they will spend on a new product in the coming year. Fourth, the way in which a question is asked sometimes influences the response.

For example, of 764 people who were asked whether they agreed or disagreec with the statement "Advertising often persuades people to buy things the) shouldn't buy," 76% agreed, 20% disagreed, and 4% had no opinion, while ol 772 people who were asked a similar question about the statemeni "Advertising seldom persuades people to buy things they shouldn't buy," 40% agreed, 56% disagreed, and 4% had no opinion

Although the way a question is asked may influence the answer, asking thE same question repeatedly can reveal changes in respondents' attitudes ovez time. The president's "report card" is generated by asking respondents periodically to assess the president's performance. As long as the question remains the same, a sharp drop in performance rating (if it cannot be explained as mere sampling error) is a good indication that the population at large has become less satisfied with performance. But a sharp drop that coincides with a rewording of the question may be nothing more than an indication of response bias.

Representativeness

A sample should be representative of the population from which it is drawn. The easiest way to ensure representativeness is to take a sample in which every member of the target population has an equal chance of being included. Such a sample is called a random sample. To obtain a truly random sample, you must have a complete list of every member of the target ~ population and you must select members from the list by a process that gives each member the same chance of being included in the sample. The random process may entail drawing names from a hat, using a table of random numbers, or telling a computer to select observations at random. If your target population is a group of people, collecting a sample entails one more step: you must track down the selected members and convince them to answer your questions. Obtaining a truly random sample is a time-consuming and difficult process. Often much more informal procedures-stopping people on the street, or in shopping malls, or in airports-are used.

Even when a serious attempt is made to obtain a random sample, the people who end up in the sample may not have been chosen in the prescribed manner for a variety of reasons. For example, in the United States, target populations are often based on the decennial census, which gradually becomes outdated, and which falls short of complete accuracy even when current. Door-to-door sampling is plagued by not-at-homes and refusals to be interviewed. 12 Telephone surveys are limited to families with listed telephones, and even many of these cannot be contacted and of those who are reached, many refuse to reply. Nonresponse in mail surveys typically runs around 80%. Thus, even with the best of intentions, it is generally not possible to obtain a random sample from the population. 13
Many polling organizations and market research agencies, nevertheless, report sample results in the form of estimates, confidence intervals, and significance tests, as if the samples that provided the data were truly random. Are such results seriously misleading? Sometimes they are. 14 But before throwing the inferential baby out with the nonrandom bathwater, it is important to understand what may go wrong when one tries to make inferences from such a nonrandom sample.

The implications of a nonrandom sample depend on the reason for the lack of randomness and the nature of the study. If you are asking about evening television viewing habits, people who are not at home in the evenings are likely to be very different from people found at home at that time. But if you are interested in which of two spaghetti sauces they prefer, it may be more reasonable to believe that at-homes can be treated as nearly, if not exactly, representative of not-at-homes. It is sometimes possible to test such a proposition by making special efforts to track down a sample of not-at-homes, but one is always left with the lingering doubt that the trackable not-at-homes are different from those who could not be tracked. And it is even harder to test whether respondents who willingly answer an interviewer's questions are representative of those who slam their doors in the interviewer's face, or hang up the telephone.

One corrective action commonly taken by sampling organizations is to replace randomly selected nonrespondents with respondents who have identical, or nearly identical, demographic profiles, such as age, gender, ethnicity, education, and income. To the extent that these demographic characteristics help to distinguish the responses of different segments of the population, this replacement technique makes sense: for example, if the young and the old have markedly different attitudes towards rock music, then replacing a young nonrespondent with an old substitute would distort your sample estimates about the population's musical tastes, while replacing the young nonrespondent with a young substitute would avoid that particular kind of distortion. Nevertheless, the replacement necessarily differs from the nonrespondent with respect to willingness to respond, and to the extent that one can hypothesize a link between willingness to respond and musical taste, a problem remains. If no such link seems plausible, then this corrective action is probably sufficient to render inferences about the population reliable.