Differences in the gut microbiome are being linked with more and more types of chronic diseases. Everything from immune disorders to mental health conditions are showing some link to alterations in the microbiome, with great anticipation that the research may reveal how the gut microbiome plays a causal role in some of these conditions. But is this anticipation warranted and where is the field going? According to a new paper published in Cell, co-authored by Microbiome Insights co-founder Brett Finlay, those in the microbiome field need to take a sober look at experimental design, statistical analysis and potential biases.

It is important to first look at why the microbiome is being implicated in so many conditions. The usual clue is a dysbiosis or deviation of microbiome composition from a so-called healthy microbiome, found in patients suffering from one condition or another. However, as Finlay and co-authors point out, dysbiosis is a loosely defined term and what constitutes a, “healthy microbiome” is even less defined. Because of this, healthy is often used to describe a control individual without the condition, so whether any differences between patient and control microbiome are the cause of the condition, consequence of the condition, or the result of some third factor is unproven. So far, the best causal evidence for the role of the microbiome in disease is the therapeutic effect of fecal microbiota transplantation in those suffering from recurrent Clostridium difficile infection.

The problem of assigning causality to observations in science is not a new one but it is important that the methods and experiments used to determine it are rigorous. Which brings us to the next major point the authors address, the use of human-microbiota associated (HMA) rodent models.

So, what are HMA rodent models? HMA models are commonly used to make causal inferences, and involve the transplantation of fecal microbiome communities from individuals with and without disease into germ-free rodents. Next, investigators perform comparative analysis of the pathologies exhibited in each group. While this is an excellent approach and is the current gold-standard, there are limitations. For example, not all taxa in the human microbiota can survive and colonize even a germ-free rodent, and those that do may not engage in the same host-microbiota interactions forged by evolution within a non-native host. More importantly, ecological factors like diet and lifestyle (among others), which might have caused dysbiosis in the first place were not present in the rodent, meaning it’s unlikely that disease-associated microbiota alterations will be replicated.

Of course, any field using rodent models to study human conditions will have caveats associated with the translation of results. Animal models are important for moving the field forward, but, as the authors point out, careful considerations of their limitations must be made. “In this review, we argue that the exceedingly high proportion of positive studies making causal claims from the use of HMA rodents is implausible and likely stems from a combination of insufficient rigor in experimental designs, inappropriate statistical analyses, and bias.”

The authors take a close look at the overwhelmingly positive results of HMA studies in the literature. In a systematic review they found that 95% of the studies they looked at concluded that fecal transfer from a diseased donor resulted in at least one disease phenotype of the human condition being greater in the mice. This extremely high success rate of a relatively new technique used across a broad range of fields should be scrutinized. The authors found that only 63% of these studies tested for a dysbiosis in the original human samples, and only 29% confirmed that some aspect of this dysbiosis was also observed in rodents. Along with this lack of mechanistic insight into the role of the microbiome in disease, the techniques and statistical analyses are not yet standardized across the field.

In HMAs there is usually only a small number of donors used and these “donor microbiomes” are replicated in a larger group of rodents. Unfortunately, this fails to capture the extensive inter-individual variability in the human microbiome, which is often larger than the effect sizes caused by disease states.  Most of this variability cannot be attributed to known factors; therefore, such small sample sizes are unlikely to be representative of any causal mechanisms in human disease. The gut microbiota transfer from a single human donor into many mice also leads to a form of pseudoreplication which artificially inflates sample size and the chance for false positives. This practice of ‘artificial replication’ was found in 84% of the papers they looked at.

Finlay and colleagues point to another common problem that plagues all fields of research, the bias towards only publishing positive results. While it’s possible that some groups have tried to replicate key studies in the field with HMA models, the majority of studies accepted to major journals are positive and the pressure to publish in such journals leads to positive-results bias. Any form of bias that creeps in needs to be addressed and if microbiome research is to be translated into clinical practice proper confirmation of causal host-microbiota relationships is needed. For this, the authors make several detailed recommendations regarding the specific issues in experimental design, analysis and bias.

They also lay out some guidelines and potential future avenues of research. First, the use of HMAs are better suited to study metabolic host-microbiome interactions and are less reliable when investigating immune-related conditions. The use of rodents that have been engrafted with human immune or metabolic cells is one way to improve these models, but the field might also consider the use of animal models that better mimic human physiology. Another promising avenue is the use of novel statistical approaches, such as Mendalian randomization and mediation analysis, which can better establish causal links in humans. Mathematical and machine learning approaches like causal molecular networks, structural equation modeling, or Bayesian network inference might also play a role.

Whatever the method or approach there will always be caveats and the point of this perspective piece was to address these, not to discredit the work already done. Everyone from the researchers to funding bodies to journal editors will need to be involved in this drive toward greater rigour in the microbiome field. Without self-correction and a changing of mindset and policies, the immense promise of this research cannot be fully realized.