Open Science practices and publish or perish dilemmas

Leah Maizey, Loukia Tzavella, David Mehler, Chris Allen.

In order to survive in the modern scientific environment researchers need to publish. Jobs, grants and opportunities all hinge on publications, which impacts acutely upon early career researchers (ECRs), such as PhD students, who do not tend to have guaranteed employment. How is this publish or perish dilemma impacted by developments in research such as those offered by Open Science (OS)? OS presents an enormous shift in the way science is conducted and comprises a range of practices that promote transparency and reliability of research. These include data sharing and study preregistration. Although OS can offer new ways to publish, considerable resources are often required to complete studies using OS methods, that can limit or delay publications. At present, formal recognition of such efforts is sparse and the pressure to publish may be greater for ECRs who engage with OS. We discuss this challenge in light of different OS methods from our perspective as ECRs at different stages.

Open data, code & materials

Efforts required to adopt OS practices vary substantially, as do their effects on publications. We have found it relatively straightforward to make research objects, such as code, data and materials openly available and strongly encourage it where possible.  We recommend making preparations from the start of projects, especially regarding data curation and legal rights to disseminate.  

Preregistration

Markedly more demanding is the adoption of preregistration, which involves a public declaration of methods and hypotheses before data collection. Hypotheses are only really hypotheses before the data is looked at. At the heart of preregistration lies the distinction between confirmation and exploration, with an emphasis on apriori methods and predictions. This distinction is not always explicit and researchers often explore data after collection. Maintaining ambiguity between confirmatory and exploratory research could convey advantages if researchers paint easily made data observations as planned tests of hypotheses to create  a publishable narrative with positive results (HARKing). We don’t suggest there is anything wrong with exploratory studies. Well-defined research (e.g. replications) may suit preregistration better than complex or qualitative questions that can be more compatible with an exploratory approach. Nonetheless, we encourage academics and journals to enforce an explicit distinction between confirmatory and exploratory research. In our experience, the largest cost of rigorous preregistration is time due to preparing experiments, anticipating outcomes and developing analyses before data collection. This can lead to fewer (but potentially higher quality) publications.

Registered reports

OS practices can be combined into a relatively recent publication format, the Registered Report (RR). RRs involve preregistration and peer-review of study protocols in eligible journals (>200 journals at present) and offer guaranteed publication irrespective of findings. Indeed, null findings appear to have a higher chance of publication with these methods. While the guarantee is beneficial, researchers should be aware of potential pitfalls. Successful RRs require extensive planning, piloting and development of analyses, as well as time to prepare and gain acceptance. Collectively, this may exceed the duration of research or training positions. While the initial peer-reviewed protocol can be included in an academic CV, we believe that the efforts required to complete a high-quality RR should be more broadly recognised by employers and funders. 

Addressing the publish or perish dilemma 

OS research practices might help to address the publish or perish dilemma for ECRs, if alternative ways of publishing are promoted. For example, OS practices allow ECRs to demonstrate productivity and research skills using preregistrations and RRs before final publication. Even code and materials are citable if researchers assign them digital object identifiers, ensuring recognisable research outputs. We make the following recommendations:  

1. Feasible science and support for ECRs

OS practices are typically resource heavy and can present obstacles for ECRs who are required to produce publications within limited time. For preregistered science, including RRs, compromises should be drawn between feasibility and rigor. To help overcome some challenges ECRs can seek out training opportunities and research initiatives. We encourage supervisors and institutions to support OS practices (e.g. via data sharing guidelines, training in reproducible programming and preregistering). 

2. Recognising OS efforts in employment

Acknowledgment of OS efforts may vary between disciplines, countries and institutions and so credit gained for these efforts may not always be transferable. Formal recognition of OS endeavours in job opportunities (e.g. desirable criteria) may help.

3. Integration of OS practices in grant applications

Funding calls often require a strong publication record. Robust preregistered science is slower science, which can result in fewer peer-reviewed publications. OS methods would be more readily adopted if encouraged by funding bodies (e.g. requirements for data sharing or study preregistration, where possible). Also, funding panels should focus on the quality rather than the quantity of research outputs, as already implemented by some. 

Our message

OS practices can result in reproducible and robust findings, which has enormous advantages for the advancement of scientific research. However, while OS developments are orientated towards the collective good, the costs tend to be borne by individual ECRs - this presents an imbalance. We strongly encourage journals, funding bodies and institutions to promote and reward ECRs who engage with OS practices so that they are not further burdened by the publish or perish dilemma.