What are some best practices for pre-registering and reporting your experimental design and analysis?

1 Why pre-register your experiment?

Pre-registration can help you avoid some common pitfalls and biases that can undermine your causal claims, such as p-hacking, HARKing, and selective reporting. By pre-specifying your research questions, hypotheses, variables, sample size, randomization, and analysis plan, you can reduce the risk of data-driven decisions, post-hoc rationalizations, and cherry-picking results. Pre-registration can also help you increase the credibility and trustworthiness of your findings, as well as facilitate peer review and replication.

2 How to pre-register your experiment?

There are several platforms and templates that you can use to pre-register your experiment, such as the Open Science Framework (OSF), AsPredicted, or the Registry of Efficacy and Effectiveness Studies (REES). Depending on your field and discipline, you may need to follow different guidelines and standards for pre-registration, such as the CONSORT statement for randomized controlled trials, or the PREPARE guidelines for animal research. You should also consider the level of detail and specificity that you want to provide in your pre-registration, as well as the possibility of updating or modifying it in case of unforeseen circumstances.

3 What to report in your experiment?

Reporting your experimental design and analysis should include providing enough information and context for your readers to understand, evaluate, and reproduce your research. You should follow the reporting standards and checklists that are relevant for your field and type of experiment, such as the ARRIVE guidelines for animal research, or the STROBE statement for observational studies. You should also report any deviations from your pre-registered plan, any exploratory or secondary analyses, any limitations or assumptions, and any ethical or practical issues that may have affected your experiment.

4 How to report your experiment?

There are different formats and outlets that you can use to report your experiment, such as journals, preprints, repositories, or blogs. Depending on your target audience and purpose, you may need to adapt your style, language, and structure of your report. However, some general principles that can help you report your experiment effectively are: use clear and concise language, avoid jargon and technical terms, provide visual aids and summaries, cite your sources and data, and acknowledge your contributions and conflicts of interest.

5 How to improve your experimental design and analysis?

Pre-registering and reporting your experiment are not the only steps that you can take to improve your experimental design and analysis. You can also seek feedback and collaboration from other researchers, conduct pilot or replication studies, perform sensitivity or robustness checks, and use open and reproducible tools and practices. By doing so, you can enhance the quality and rigor of your research, as well as contribute to the advancement of knowledge and science.

6 How to learn more about experimental design and analysis?

If you want to learn more about experimental design and analysis, especially for causal inference methods, there are many resources and courses that you can access online or offline. For example, you can check out the online courses offered by Coursera, edX, or Udemy on topics such as experimental design, causal inference, or randomized evaluation. You can also read books, articles, or blogs that cover the theory and practice of experimental design and analysis, such as Experimental Design for the Life Sciences by Ruxton and Colegrave, Causal Inference: The Mixtape by Cunningham, or The 20% Statistician by Lakens.

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?