Sampling techniques

In an ideal world, researchers might have enough time, money and research assistants to include census-like studies of entire populations in their research designs. In reality, this is entirely implausible, but luckily statistics implies that this is also entirely unnecessary. Sampling is much like the logic of flipping a coin – if you only flip it six times, the chances of getting an accurate 50/50 distribution is diminished, but if you flip it one hundred times you’ll probably get something a lot closer to 50/50. In the same way, analyzing an entire population only requires a researcher to study a large enough chunk to ensure that their analysis is not significantly influenced by random chance and errors.

Generalizability is often considered to be a paramount goal in scientific inquiry, although this is not entirely the case in social sciences. Generally speaking, generalizability is a typical goal of quantitative analysis in the social sciences, but somewhat less so for some instances of qualitative analysis. A typical line of reasoning for not striving for generalizability might be that access to the population that is being studied is limited and a representative sample could never be selected; researchers with constructivist viewpoints might also conclude that generalizability is an impossible goal to achieve. That said, even when generalizability is not a plausible goal, the specific sampling technique used is still important.

When generalizability is a plausible goal, the sampling techniques used are termed as “probability sampling methods.” By utilizing probability mechanics, these sampling methods avoid systematic biases caused by underrepresentation and overrepresentation. The most obvious of these methods is simple random sampling, where subjects are selected with a random numerical component to ensure that selection is entirely random. Closely related to this sampling technique is systematic random sampling, where specific items in sequential files are selected according to the necessary representative sample size and the total number of available entries. If it is essential for a researcher to make sure their sample accurately represents a specific distribution of a characteristic in a population, proportional stratified sampling is used; similarly, if a researcher wants to focus on a specific (or underrepresented) characteristic in a population, disproportionate stratified sampling, in combination with weighting techniques, is used. Finally, researchers can use cluster sampling as an alternative to simple random and systematic random sampling techniques. Cluster sampling focuses on random selection of naturally clustering elements of a population (such as city blocks or schools), and while it is sometimes convenient for survey researchers, it is unfortunately more likely to cause sampling errors.

When generalizability is not a plausible goal, nonprobability sampling is used. The most obvious type of these sampling techniques is availability sampling, where the sample is simply selected by convenience (such as people passing by on the street). The use of this technique varies contextually, but given the difficulty of ensuring that all subjects being studied belong to a particular population, it is not often applicable to nuanced research questions. More preferable techniques include quota sampling and purposive sampling. Quota sampling represents an attempt to mitigate the detrimental effect of nonprobability based sampling methods – researchers attempt to meet sampling quotas that represent a population, but since all possible characteristics of a population can’t realistically be identified, this sampling technique still fails to lead to a generalizable sample. Purposive sampling techniques are used when the sample is selected to fulfill a specific research purpose, such as only interviewing people who are particularly knowledgeable about a subject or the study of a small subset of a population. Finally, there is snowball sampling, where each subject provides the researcher with a list of other cooperative subjects – this is best utilized when studying populations with limited access, such as police officers.


CITI reflections

Perhaps the fact that a mandatory online training course exists should have tipped me off, but it was still surprising how very bureaucratic institutional review board (IRB) procedures appear to be. That said, I think this is a bureaucracy that we are better with than without. While some standards may appear onerous, IRB boards don’t only serve to protect scientists from ethical concerns, but their stringent procedures may also actually help restore some of the faith that the public has lost in science.

In response to ethically questionable experiments, such as the Milgram shock experiment, the Stanford prison experiment, and, most notoriously, the Tuskegee syphilis trials, the federal government issued the Belmont Report, which ultimately led to the establishment of numerous federal research guidelines and the establishment of IRBs at universities and other research institutions. While the premise of scientists policing themselves might not appear to be a trustworthy venture at face value, this arrangement is more nuanced than it might appear. IRBs not only require the presence of a member who is not associated with the institution, but also the presence of a non-scientist member. Furthermore, efforts must be made to ensure that IRBs are not representative of only one gender. The diverse membership that these procedures create help ensure that IRBs respect multiple perspectives when reviewing proposals.

There are numerous ethical concerns that IRBs are concerned with, but they’re broadly concerned with the three recommendations of the Belmont Report: respect for persons, beneficence and justice. IRBs primarily ensure respect for persons through high standards of informed consent – by ensuring that participants in research understand the aims and potential risks of the study. One of the most appalling aspects of the Tuskegee syphilis trials was the fact that the participants were unaware they were participating in research, and unaware of the disease that was being studied, so powerful informed consent procedures can help safeguard from such an incident occurring again (also of importance: IRBs enforce several extra precautionary measures when dealing with medical research). Beneficence, defined as the normative goal of maximizing potential benefits and minimizing harm, ensures that potentially harmful research is actually worth doing and not only of interest to a handful of scientists. One of the frequent criticisms of the Milgram shock experiment was that it potentially harmed the research subjects while failing to adequately address a worthwhile research question. Milgram said that the goal of the experiment was to understand if obedience could explain Nazi war atrocities, but critics rejected the study as unable to adequately address this research question. Finally, there is the issue of justice, which, in the context of the Belmont Report, strives to ensure that potential harm is distributed equally and fairly across research participants and populations. This concept is exemplified well in the previously discussed examples of IRB concerns, but one additional way that justice is ensured is by applying special protections to at-risk populations such as youth and the incarcerated. Special consideration must be given not only to those who are legally unable to consent, but also to those who may be coerced into participation in research through various mechanisms that researchers cannot control.

Some might be concerned that these protections are too prohibitory, and that powerful research cannot be achieved anymore. This is where IRB deliberation is often most important. IRBs help determine when the use of deception is worthwhile and appropriate, as well as waiving informed consent procedures when necessary. Conversely, a lot of research (such as survey research) is entirely benign and, therefore, criteria exist for expediting or bypassing IRB review when appropriate. As a researcher in the social sciences, I would expect my work to fall in the latter category, but I endorse the idea of ensuring that all researchers are thoroughly educated in ethical matters anyways.

Privacy Statement