Our experience with the Registered Report format

Jul 29, 2019
3
0

By Joyce C. He and Stéphane Côté 

We are honored to contribute one of the first two Registered Reports (RRs) published in Nature Human Behavior, "Self-insight into emotional and cognitive abilities is not related to higher adjustment", published online a week ago (link to the article here). Here we explain why we decided to do this project as an RR and the main things that we learned along the way.

We decided to try the RR format for the first time when around the same time:

  • We learned that Nature Human Behavior started welcoming these types of contributions
  • We attended a session on pre-registration and RRs at the Society for Personality and Social Psychology conference that addressed many of our questions about the format
  • We were planning studies on self-insight and adjustment that seemed important for both theoretical and practical reasons.

The RR format seemed ideal because we felt that any result—including any non-significant result—should be published. There had been multiple competing perspectives proposed in existing literature regarding the benefits of self-insight for adjustment. While each hypothesis that we thought of testing seemed plausible, there were compelling reasons to doubt each of them, so we thought the results would be important even if no hypotheses were supported. We did not want to find ourselves in a typical situation where editors might be less inclined to accept our paper if we did not have significant results.

The following list captures what we were most surprised to discover about the RR format as we became engaged in the various stages:

  • RRs are an efficient use of time. We did not write the results and discussion sections until we knew we would publish our study in a widely-read journal (assuming that we did not deviate from our proposal). Also, we were not permitted to make any changes to the introduction in Stage 2. This was welcomed, given that in the past, we had spent lots of time re-writing introductions sections prior to resubmissions.
  • Although at the outset it seemed risky to write a proposal that could be rejected, reviewers had the opportunity to make suggestions to improve our study rather than recommend rejection because of its limitations. One of us has since reviewed RR submissions and found it easier to recommend revisions knowing that authors had the opportunity to improve their study before conducting them.
  • It is not a myth that the RR process results in better studies. This is the review process we have both learned the most from. This was especially striking for the second author who has submitted over 100 manuscripts in his career so far. We received multiple rounds of comments by researchers who are true experts in this area and open science. We feel this likely would have never happened outside of the RR process.
  • We were encouraged by the positive and constructive tone of the reviews. The process was not easy—we went through four rounds of reviews at Stage 1 and two rounds at Stage 2. But, the process felt like a collaborative effort between reviewers, editor, and authors in crafting the final product rather than a confrontation. We were pleasantly surprised that all reviewers signed their reviews.
  • RRs are an efficient use of funds. We did not pay for the main study until we knew we would publish the study (assuming, again, that we did not deviate from our proposal). This was an expensive study and we would have hesitated to conduct it at all had it not been an RR.

As is probably obvious, we are sold on the benefits of RRs for scientific progress and will continue to seek opportunities to publish in this format for our future research. We will encourage editors to offer this format in the other journals we consider for publishing our work, and also encourage other researchers to try out the RR format for their own research.

Joyce C. He

PhD Student, Rotman School of Management, University of Toronto

No comments yet.