In our commitment to robust and open science, Ecological Solutions and Evidence has introduced registered reports as a new article submission type – but what exactly are they? We asked Tim Parker, Shinichi Nakagawa and Hannah Fraser, three advocates and experts on transparency and open science in ecology.

Registered reports are a new type of scientific paper designed to reduce publication and reporting bias, improve the quality of published work, and promote transparency of the scientific process.

The key difference between a typical scientific paper and a registered report is that the primary peer review for a registered report happens before data gathering.

First, researchers design a study, which may involve gathering pilot data to help refine their methods. Next, they submit an introduction and a detailed methods section as a stage 1 registered report to the journal.

This stage 1 manuscript goes through the standard peer review process of the journal, but is evaluated based on the value of the ideas and the suitability of the methods. If, after any revisions, the manuscript satisfies the reviewers and editors, the stage 1 report is published in Ecological Solutions and Evidence and the authors are offered an ‘in principle’ acceptance decision for the full study (note that not all journals publish the stage 1 manuscript as a separate article).

This acceptance means that as long as the authors implement the study as planned within the specified timeline, the journal agrees to publish the results regardless of the outcome. Thus when authors submit the completed manuscript at stage 2, the review process only involves assessment of whether the originally proposed methods were followed, the results are fully reported, and the discussion aligns with the results.

Let’s review the benefits of this publication model.

---

1. Because acceptance for publication does not depend on the outcome of the study, registered reports reduce publication and reporting bias. Readers of papers published as registered reports should have elevated confidence in the results. As registered reports become more common, the overall reliability of the literature will also be improved. Of course the degree to which bias is reduced by registered reports depends on the degree of publication and reporting bias as well as other questionable research practices in the subfield in question. Unfortunately we don’t have a precise estimate of these biases in ecology, but we do have compelling evidence that they are real.

2. The quality of methods in registered reports should be higher, on average, than the quality of methods from unregistered studies. The high quality derives from peer review prior to study implementation, when peer review still has the opportunity to improve methods of experimental design or data gathering, unlike peer review of traditional manuscripts. This is a win for everyone involved – researcher, reviewer, editor and reader.

3. Researchers can implement a study without worrying if results will be sufficient for publication. This may be particularly valuable to early career researchers who rely on publications as they compete for jobs and grants.

---

This is not a comprehensive discussion of the benefits to registered reports, but we believe that any one of these three listed benefits is sufficient reason to adopt this publication model.

Answering concerns about registered reports

Researchers may sometimes be reluctant to submit a registered report for any number of reasons. Although registered reports will not be appropriate for every situation, they are likely appropriate for substantially more situations than many researchers will assume.

---

1. In our opinion, the primary situation where a registered report will not work is when a time constraint prevents pausing for peer review prior to an impending field season. However, with sufficient lead time, and possibly also with expedited peer review facilitated by journals, this problem can often be avoided.

2. Many perceived obstacles boil down to a desire or need for flexibility in working with data. Fortunately, whether you work with existing data; need to see data prior to finalising an analysis; want the option to conduct follow-up analyses; or focus on discovery or exploration, most work can be accommodated by, and benefit from, the registered reports model.

3. Further, concerns about extra work are mostly misguided. The main outcome will be a shift in the timing of work, with more planning and preparation occurring before data collection, and less worked out during the course of data collection and analysis.

4. Finally, for those who may argue that pre-registration (a distinct process that involves public archiving of a research plan prior to conducting the work, but submission of the manuscript through regular channels after work is complete) is sufficient and easier, we acknowledge the benefits of pre-registration, but we also call attention to the three benefits highlighted above, none of which are produced by pre-registration.

Please see our recent article for a more thorough consideration of these issues.

---

Registered reports are still rare in ecology, but between the three of us, we have three different registered reports that have made it past stage 1. We can attest that it takes some getting used to; waiting for peer reviews, possibly for months, before beginning research can be frustrating.

However, we can also attest that obtaining in-principle acceptance is remarkably satisfying. Once we crossed that hurdle, we knew that, as long as we did what we said we would, the publication could not be derailed by an unexpected result or a marginal p-value.

We are convinced that registered reports have an important role to play in the future of ecological research, and we are pleased that Ecological Solutions and Evidence is piloting registered reports for the British Ecological Society.

Would you like to submit a Stage 1 Registered Report to Ecological Solutions and Evidence? See if your article fits within our journal’s scope and submit here, or find out more in our Registered Reports Author Guidelines.

---

Biographies

Tim Parker, Whitman College
Much of Tim’s recent and ongoing work explores the degree to which scientific results are reliable and the factors influencing reliability.  This empirical work has informed his efforts to promote the adoption of practices, such as registered reports, designed to improve the reliability of evidence in science.

Shinichi Nakagawa, UNSW Sydney
Shinichi’s research group is interested in research synthesis, especially of evolutionary and ecological literature. Research synthesis has naturally led to meta-research quantifying bias and gaps in our knowledge.

Hannah Fraser, University of Melbourne
Hannah Fraser is President of the Association of Interdisciplinary Meta-Research and Open Science. She is an ecology meta-researcher working at the University of Melbourne as part of the interdisciplinary meta-research group with a passion for learning how to make ecology research more reliable.