Photo: Slide from Vivien Tseng's talk at the AESIS conference, showing that the pathway from Research to Policy to Outcomes is not a straight road, it’s an LA Freeway.
Typically editors go to conferences to learn about the newest research coming out in a field, and to talk to authors face-to-face about what we look for in the papers we publish and our peer-review process. The AESIS (Network for Advancing & Evaluating the Societal Impact of Science) conference on The Impact of Social Sciences and Humanities for Society started where our conference interactions usually end: instead of focusing on how research papers are assessed from submission to publication, its starting point was how the impact of these papers should be assessed *after* they are published.
Assessing societal impact is no easy feat. Everyone at the conference agreed that we need to move beyond bibliometrics, because citations only tell us that a piece of research was referenced, not that it was actually *used*. Bibliometrics are also biased to imply that more is better. But as Síle Lane from Sense about Science noted, the *number* of users doesn’t matter if they are the *right* users. Moreover, sometimes impact means convincing stakeholders *not* to do something - how do we measure what’s not there? There is also an unknown time lag between publishing a piece of research and practical application, so it’s not even possible to establish with certainty *when* to measure impact. Vivien Tseng described efforts by The William T Grant Foundation to address some of these questions in their call for proposals on research on when, how, and under what conditions research evidence is used to improve outcomes for young people.
Another point of universal agreement at AESIS was that developing relationships with stakeholders is a key ingredient for impact. For instance, understanding what stakeholders want and need before embarking on a research project ensures that the study asks the questions that relevant users want answered. Avi Green from Scholars Strategy Network could not emphasize enough the importance of researchers engaging with policymakers face-to-face, and building trust, to get their research seen and ultimately used. Unfortunately, these are areas where journal editors are simply not positioned to support researchers. We enter the process after research has already been conducted, and our roles do not naturally lead to highly developed networks of policymakers, and certainly not across every country represented in the research we publish. So what *can* we do?
At Nature Energy, we've spent a lot of time thinking about and discussing this problem: we publish research that is policy-relevant, but academic research papers are written for other academics, making them difficult to interpret and use for policy audiences. Our conversations culminated in the new Policy Brief format that Nature Energy launched earlier this month. Policy Briefs are written by the authors of a research study published in the journal, and provide a short (no more than two pages) high-level take on the research and its key policy messages.
As editors our expertise is in communicating research within research papers themselves, and in producing summaries aimed at a broader—though largely academic—audience. So communication for policy is a new endeavor for us, and we are grateful to individuals at the Government Office for Science and the Parliamentary Office of Science and Technology (POST), both in the UK, who took the time to give us feedback on the structure we developed. Indeed, some of their suggestions—like putting the results before the description of the study—made us editors downright uncomfortable! Thanks also to our first round of guinea pigs (i.e. authors of papers published in Nature Energy in 2018) who worked through various iterations of our author templates and instructions to ultimately produce our first set of published Policy Briefs.
We view Policy Briefs as a work in progress, and we welcome your feedback so that we can adapt and modify the template as needed. We’re especially interested to hear from their intended audience: if you’re a policy professional and would like to know more about these articles or have suggestions for us on how to make the most of this content type we want to hear from you! If you know someone who is a policy professional, please share the editorial that introduces the format in more detail with them.
The next step for us is assessing the impact of our Policy Briefs, to determine if they’ve had the desired effect of better connecting the research we publish to the policymakers that can use it. This brings us right back to all the thorny issues raised at the AESIS conference - what metrics make sense? How do we determine that we’ve reached the right people rather than the most people? What is a reasonable timescale to measure impact?
The pathway from research to policy to outcomes is complicated, involves a range of stakeholders, and is influenced by many factors outside of any single actor's control. We are excited to be a more active contributor to this process.