Reading and Interpreting Scientific Literature

CHAPTER 9 Reading and Interpreting Scientific Literature



Philip H. Kass


The tremendous growth in the number of medical and scientific journals published in the past 20 years, in both printed and electronic formats, coupled with practitioners’ ability to provide increasingly higher standards in veterinary medical care, places a growing responsibility on veterinarians to avail themselves of the medical literature to stay abreast of current developments. However, with this responsibility comes the expectation that veterinarians have the skills and ability to evaluate critically a manuscript’s accuracy and value. It might be easy to contend that such qualities are not essential because of the integrity and rigor of the peer and editorial review process and to believe in the infallibility of these established procedures for ensuring scientific oversight. This contention is naïve and incorrect.


With vast amounts of new information constantly entering medical literature, it is undeniably tempting and an apparent efficient use of time for a practitioner, upon reading a paper, to accept unquestionably the entirety of the findings or, worse, to accept the limited information absorbed from perusal of the abstract. The methods section of a manuscript may be more overlooked than any other portion, yet that section often holds the key to aiding the reader’s ability to discern between articles with reliable conclusions and those that should be dismissed. This problem is compounded as medicine becomes more sophisticated. Earlier generations of veterinarians can lack familiarity with advances that successive ones become exposed to during their formal education, and it is unrealistic to expect that all readers have the knowledge to discern between valid and invalid findings in their interpretation.


Nevertheless, certain fundamental issues are common to all scientific manuscripts. Although these issues are not new, they are even more relevant now than in the past because of continued development of medical interventions to combat disease, advances in modern experimental and nonexperimental study design, and the flourishing of classic and novel statistical methods in readily accessible computer software. Although it is not feasible to proscribe a single systematic method of evaluating each manuscript because of the myriad of complexities unique to each manuscript and the absence of some objective weighting scheme to score quality, attention to the issues that follow should guide readers in navigating some of the most common, yet least appreciated of steps to ensure the validity of a study.



CONFOUNDING AND THE ABSENCE OF GROUP COMPARABILITY


One of the most stringent criteria for validity in controlled studies is the requirement for group comparability. Although this has often been interpreted to mean that individuals in a control group (e.g., a placebo or sham group) should be as much like individuals in a treatment group as possible, it is both possible and desirable to be more precise in definition. Instead, it is preferential to recognize the incidence of some medical outcome in a control group as a surrogate for what the incidence of the same outcome would have been in a treatment group had the latter never been treated. The absence of confounding—and hence of group comparability—is predicated on the assumption that these two incidences are identical.


In an experimental design of comparative treatment efficacy, such as a clinical trial, the prototypical approach to achieving comparability is to institute a randomization protocol. This pertains to the random assignment of study participants to one of two or more treatment or control options and should not be confused with random selection of individuals for inclusion in a study. Under simple randomization protocols, no individual is any more or less likely to receive one treatment option than another, implying that, in the absence of confounding, any individual’s response to a control treatment—either hypothetical or observable—is independent of what that individual’s assignment actually was. Unfortunately, randomization is not a guarantee of group comparability: the smaller a study, the greater the probability of chance assignment imbalances that could lead to confounding. Conversely, the larger a study, the less likely it is that such imbalances will occur. For example, if a horse’s sex is a confounder, in a randomized study of four horses, there is a 50% probability that each group will have two horses of the same sex. However, in the same study of 100 horses, the probability that each of the two groups will have 30 or more horses of one sex (instead of the expected 25) is only approximately 16%, and the probability of having 35 or more horses is less than 1%. When confounding from known chance imbalances does occur in randomized studies, the bias can often be removed in the statistical analysis. Such imbalances from unknown or unmeasured confounders, however, cannot analytically be removed and result in residual bias in the study.


The problem of confounding has more substantial repercussions in manuscripts in which treatment effects in a nonexperimental (i.e., nonrandomized) study were compared. Although treatment is randomly assigned in an experimental study, in a nonexperimental study, the individuals are purposefully (nonrandomly) given a particular treatment. For example, a certain treatment may be the only option available for a time but may be superseded by a different treatment at a later time. A study in which these treatments were compared would be unable to distinguish between their alleged effects and effects related to time (such as from secular changes in disease severity, improved standards of care and veterinarian proficiency at treating cases over time, or different clinicians handling the cases at different times). Another example arises when clinicians prescribe treatments based on their perceptions of severity of illness: one treatment may be recommended for milder cases, whereas a novel treatment may be reserved for only the most refractory cases. It would be difficult to distinguish between treatment effects and those related to severity or other factors that motivated a veterinarian’s treatment choice. This type of bias is referred to as confounding by indication, and it can potentially affect all nonrandomized studies of treatment efficacy.

Related posts:

  1. Protecting Yourself with Medical Records
  2. Systemic Chemotherapy for Oncologic Diseases
  3. Coagulopathies
  4. Nutritional Factors in Developmental Orthopedic Disease

Stay updated, free articles. Join our Telegram channel
Tags:
May 28, 2016 | Posted by in EQUINE MEDICINE | Comments Off on Reading and Interpreting Scientific Literature

Full access? Get Clinical Tree
Get Clinical Tree app for offline access