Slow and steady: our meta-analysis on greater male variability in academic performance

By Rose O'Dea and Malgorzata (Losia) Lagisz

Go to the profile of Rose O'Dea
Sep 25, 2018
0
1

The paper, authored by us (Rose and Losia) and our co-authors Michael and Shinichi, is freely available in Nature Communications: https://bit.ly/2N2ymie

The seed for this paper was planted five years ago. Michael was sat at Losia and Shinichi’s kitchen table in Dunedin, New Zealand, listening to Shinichi describe a new meta-analysis technique he’d been developing. This technique could test for differences in variability between two groups, while controlling for differences in means. Michael suggested we use this technique to test the greater male variability hypothesis.

It would be easy. Losia found a recently published meta-analysis that tested for gender differences in average school grades; we’d do a re-analysis. It would be a simple project for Michael’s honours student, Rose, to learn some meta-analysis basics; she was joining Shinichi’s new lab in Sydney, Australia, at the beginning of 2015. The paper would be finished (we all thought) within a year, before Rose started a PhD. But it took that long just to get the data, never mind analysing and writing up our results.

over 1500 days between project conception and publication

The descriptive statistics of studies included in the original meta-analysis were not available. So, we downloaded the papers and re-extracted the necessary data, and contacted the authors of papers that hadn’t reported descriptive statistics. We also updated the original systematic search to find papers published since mid–2011. Losia and Rose each screened over 3,000 abstracts (thanks abstrackr), and by the start of 2016 we finally had our comprehensive dataset.

But by then, this was a side-project. Not part of anyone’s grant or thesis, it was often relegated to the bottom of our priority lists. Having each other as co-authors was essential: the project never fully stagnated because there was someone else to keep it moving. Had we been doing the project alone it would probably lay abandoned.

We were also challenged by our ignorance of the field – we are evolutionary biologists and behavioural ecologists, not social scientists or psychologists. It took time to find, and read, studies that informed our introduction and discussion. Often we were missing jargon that was the key to unlocking this information. Here we are thankful to popular science communicators for bridging these gaps (e.g. the term ‘occupational segregation’, which appears in our introduction, was learnt on an episode of Freakonomics Radio), and to psychologists who commented on earlier drafts.

Our meta-analysis found the school grades of girls and boys were more similar in STEM compared to non-STEM subjects. Source: Losia Lagisz

In October 2017, over two and a half years since we first started collecting data, we submitted our manuscript. It was rejected without peer review. We tried a second journal, and then a third, with the same result. Without peer-review feedback we didn’t know how to improve our manuscript. Were we simply aiming too high?

To make it past an editor we jazzed up our submission with a new, livelier title and abstract, and a thoughtful cover letter that sold the context of our study and asserted the confidence in our methodology. It worked – Nature Communications sent our paper out for peer review in January 2018. Extensive revisions followed (including a jazzing-down of our abstract – curious readers can read these reviewer comments and responses for themselves). With the finish line in sight this side-project became a top priority for the first time in over two years.

In the paper’s discussion we argue that being brilliant is not the key to succeeding in STEM. To make a meta-observation about our meta-analysis, this was true for our paper. We took a seemingly simple idea conceived around a kitchen table and, while it grew into a much larger, harder, and more time consuming paper than we’d expected, we supported each other and we persisted.

Go to the profile of Rose O'Dea

Rose O'Dea

PhD Candidate, University of New South Wales, Sydney

1 Comments

Go to the profile of Jenn Richler
Jenn Richler 3 months ago

I love the timeline! It's easy to forget just how long the research lifecycle can be, and how rejection often plays a part in that. This is a great reminder (especially for early career researchers) of what the process can be like.