What is the most relevant question you should ask when reading the method section?
Reading scholarly studies can help journalists integrate rigorous, unbiased sources of information into their reporting. These studies are typically carried out by professors and professional researchers — at universities, think tanks and government institutions — and are published through a peer-review process in which those familiar with the study area ensure that there are no major flaws. Show
Even for people who carry out research, however, interpreting scientific (and social science) studies and making judgments about their quality can be difficult tasks. In a now-famous article, Stanford professor John Ioannidis argues that “most published research findings are false” due to inherent limitations in how researchers design studies. (Health and medical studies can be particularly attractive to media, but be aware that there is a long history of faulty findings.) Occasionally, too, studies can be the product of outright fraud: A 1998 study falsely linking vaccines and autism is now perhaps the canonical example, as it spurred widespread and long-lasting societal damage. Journalists should also always examine the funding sources behind the study, which are frequently declared at the study’s conclusion. Before journalists write about research and speak with authors, they should be able to both interpret a study’s results generally and understand the appropriate degree of skepticism that a given study’s findings warrant. This requires data literacy, some familiarity with statistical terms and a basic knowledge of hypothesis testing and construction of theories. Journalists should also be well aware that most academic research contains careful qualifications about findings. The common complaint from scientists and social scientists is that news media tend to pump up findings and hype studies through catchy headlines, distorting public understanding. But landmark studies sometimes do no more than tighten the margin of error around a given measurement — not inherently flashy, but intriguing to an audience if explained with rich context and clear presentation. Here are some important questions to ask when reading a scientific study: 1. What are the researchers’ hypotheses? A hypothesis is a research question that a study seeks to answer. Sometimes researchers state their hypotheses explicitly, but more often their research questions are implicit. Hypotheses are testable assertions usually involving the relationship between two variables. In a study of smoking and lung cancer, the hypothesis might be that smokers develop lung cancer at a higher rate than non-smokers over a five-year period. It is also important to note that there are formal definitions of null and alternative hypotheses for use with statistical analysis. 2. What are the independent and dependent variables? Independent variables are factors that influence particular outcomes. Dependent variables are measures of the outcomes themselves. In the study assessing the relationship between smoking and lung cancer, smoking is the independent variable because the researcher assumes it predicts lung cancer, the dependent variable. (Some fields use related terms such as “exposure” and “outcome.”) Pay particular attention to how the researchers define all of the variables — there can be quite a bit of nuance in the definitions. Also look at the methods by which the researchers measure the variables. Generally speaking, a variable measured using a subject’s response to a survey question is less trustworthy than one measured through more objective means — reviewing laboratory findings in their medical records, for example. 3. What is the unit of analysis? For most studies involving human subjects, the individual person is the unit of analysis. However, studies are sometimes interested in a different level of analysis that makes comparisons between classrooms, hospitals, schools or states, for example, rather than between individuals. 4. How well does the study design address causation? Most studies identify correlations or associations between variables, but typically the ultimate goal is to determine causation. Certain study designs are more useful than others for the purpose of determining causation. At the most basic level, studies can be placed into one of two categories: experimental and observational. In experimental studies, the researchers decide who is exposed to the independent variable and who is not. In observational studies, the researchers do not have any control over who is exposed to the independent variable — instead they make comparisons between groups that are already different from one another. In nearly all cases, experimental studies provide stronger evidence than observational studies. Here are descriptions of some of the most common study designs, presented along with their respective values for inferring causation:
5. What are the study’s results? There are several aspects involved in understanding a study’s results:
6. How generalizable are the results? Study results are useful because they help us make inferences about the relationship between independent and dependent variables among a larger population. The subjects enrolled in the study must be similar to those in the larger population, however, in order to generalize the findings. Even a perfectly designed study may be of limited value when its results cannot be generalized. It is important to pay attention to the composition of the study sample. If the unit of analysis is the individual, important factors to consider regarding the group’s composition include age, race/ethnicity, gender, socioeconomic status, and geographic location. While some samples are deliberately constructed to be representative of a country or region, most are not. 7. What limitations do the authors note? Within a research article, authors often state some of the study’s limitations explicitly. This information can be very helpful in determining the strength of the evidence presented in the study. 8. What conclusions do similar studies draw? With some notable exceptions, a single study is unlikely to fundamentally change what is already known about the research question it addresses. It is important to compare a new study’s findings to existing studies that address similar research questions, particularly systematic reviews or meta-analyses if available. Further: One hidden form of bias that is easily missed is what’s called “selecting on the dependent variable,” which is the research practice of focusing on only those areas where there are effects and ignoring ones where there are not. This can lead to exaggerated conclusions (and thereby false media narratives). For example, it is tempting to say that “science has become polarized,” as survey data suggest significant differences in public opinion on issues such as climate change, vaccinations and nuclear power. However, on most scientific issues, there is almost no public debate or controversy. Additionally, the reality of “publication bias” — academic journals have traditionally been more interested in publishing studies that show effects, rather than no effects — can create a biased incentive structure that distorts larger truths. For an updated overview, see a 2014 paper by Stanford’s John Ioannidis, “How to Make More Published Research True.” Keywords: training What is typically included in the method section of a research article quizlet?What is typically included in the introduction section of a research article? It provides the details of the methodology used in the study. What is typically included in the method section of a research article? It provides interpretation of the findings.
What is a potential problem with the method of authority?What is a potential problem with the method of authority? A person may claim to be an expert when he or she really is not. An "expert" answer may only be a personal, subjective opinion. An expert may be giving answers outside his or her area of expertise.
Which section of a research article is most likely to provide suggestions for the interpretation and applications of research results?You should make suggestions for further research in the discussion section. NOTE: Besides the literature review section, the preponderance of references to sources in your research paper are usually found in the discussion section.
Which section of a research article includes an explanation of the procedures?The Methods section of a research article includes an explanation of the procedures used to conduct the experiment. For authors of scientific research papers, the objective is to present their findings clearly and concisely and to provide enough information so that the experiment can be duplicated.
|